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PREFACE 


In  the  area  of  advertising  research,  the 
opportunity  to  develop  a  controlled  experiment 
and  implement  it  in  the  field  is  rare  indeed.  Few, 
if  any,  private  sector  companies  could  undertake 
such  an  ambitious  project  and  sustain,  over  a 
period  of  several  years,  a  research  team  and 
operational  infrastructure  to  ensure  its  successful 
completion. 

This  report  represents  the  efforts  of 
participants  from  a  broad  spectrum  of  Department 
of  Defense  (DoD)  organizations,  and  from  several 
contractors  serving  in  a  supporting  role. 
Limitations  on  space  and,  candidly,  my  ability  to 
recall  the  respective  contributions  of  all  those 
involved  in  this  four-year  project  do  not  permit 
complete  recognition  here.  Yet,  this  in  no  way 
lessens  my  appreciation  for  their  effort.  I  do 
want,  however,  to  take  particular  note  of  a  few 
individuals  who  deserve  special  recognition  for 
their  efforts. 

Representatives  in  the  Military  Services  were 
confronted  with  some  of  the  most  challenging 
aspects  of  this  experiment  It  was  their  charge  to 
execute  the  experimental  design  in  the  field  and  to 
supply  extensive  data  sets  on  the  several  measures 
used  to  evaluate  the  field  experiment.  By  one 
count,  more  than  480  separate  data  submissions 
were  required  throughout  the  course  of  this 
experiment  Their  cooperation  and  level  of 
professionalism  contributed  immeasurably  to  this 
project's  success. 

From  its  original  conception  almost  four  years 
ago,  through  completion  of  this  final  report  on  the 
field  experiment,  representatives  from  the  Office 
of  the  Assistant  Secretary  of  Defense  (Force 


Management  and  Personnel),  Directorate  for 
Accession  Policy,  provided  continued  guidance 
and  support.  I  am  especially  grateful  to  Dr.  W.S. 
Sellman,  Captain  Louise  C.  Wilmot,  and 
Lieutenant  Colonel  John  A.  Ford,  tire  project 
manager,  from  that  office.  Together,  they  served 
as  a  catalyst  in  gaining  tire  cooperation  of  those  in 
the  many  participating  DoD  organizations.  In 
addition.  Dr.  Sellman  and  Lieutenant  Colonel 
Ford  made  many  contributions  to  tire  several 
drafts  of  this  final  report  that  improved 
immeasurably  its  appearance  and  readability. 

Members  of  the  Defense  Manpower  Data 
Center  (DMDQ,  especially  Mr.  Robert  C. 
Brandewie,  Ms.  Helen  Hagan  and  Ms.  Lynn 
Prince,  provided  much  needed  technical  support 
in  processing  and  retaining  many  of  tire  data  bases 
used  in  the  analysis  phase  of  this  experiment 
Through  the  efforts  of  these  individuals,  DMDC 
will  serve  as  the  permanent  repository  of  all  data 
collected. 

In  total,  72  media  markets  (i.e.,  collections  of 
counties  forming  media  areas  defined  by  tire 
Arbitron  Company)  from  a  universe  of  214 
possible  markets  were  used  in  the  experiment  I 
wish  to  extend  my  appreciation  to  the  analysts  at 
the  RAND  Corporation,  especially  Dr.  Michael 
Polich,  in  providing  the  initial  technical  support  to 
choose  a  set  of  statistically  balanced  media 
markets. 

Miss  Phoebe  Weiner  and  her  staff  at  PEP 
Systems,  Inc.,  were  responsible  for  collecting 
and  processing  the  many  files  obtained  from  the 
Military  Services  and  their  advertising  agencies 
covering  the  media  activity  in  each  of  these  72 
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markets. 

In  addition  to  the  analytical  support  provided 
by  die  Wharton  Center,  Dr.  Ambar  Rao  and  his 
colleagues  at  OR/MS  Dialogue  deserve  much 
credit  for  their  innovative  analysis  and  insights 
into  die  findings  offered  throughout  the  many 
phases  of  this  project 

Extensive  management  and  logistical  support 
was  provided  by  CACI  International  Inc.,  and  I 
am  especially  grateful  to  Mr.  Dan  Huck,  Director 
of  Market  Analysis  of  CACI,  and  Mr.  Jerry 
Allen,  formerly  of  CACI,  for  their  extensive 
assistance.  Mr.  Huck  and  his  staff  were 
responsible  for  the  typing  and  printing  of  this 
report,  as  well  as  several  earlier  documents 
having  to  do  with  various  phases  of  the  project 


Ms.  Lynn  Lucchetti,  Director  of  die  Joint 
Recruiting  Advertising  Program  (JRAP),  was 
instrumental  in  executing  successfully  the  field 
experiment's  media  plans  developed  for  JRAP. 
Her  strength  of  personality  and  spirit  of 
cooperation  were  admirable  assets. 

Every  precaution  was  taken  to  ensure 
accuracy,  but  any  errors  in  the  analysis,  findings, 
and  interpretations  leading  to  the  conclusions  and 
recommendations  depicted  in  this  report  remain 
the  sole  responsibility  of  the  author  and  the 
Wharton  Center. 


Vincent  Carroll 

The  Wharton  Center  for  Applied  Research 
July  1987 
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EXECUTIVE  SUMMARY 


This  document  describes  the  formulation  and 
execution  of  a  DoD-sponsored  field  experiment  to 
test  the  effect  of  advertising  on  enlistments  under 
alternative  budget  levels.  It  reports  the  research 
findings  and  addresses  their  potential  budget 
implications. 

The  DoD  Advertising  Mix  Test,  implemented 
as  an  experiment  in  Fiscal  Year  (FY)  1984, 
revealed  that  those  geographic  areas  subject  to  an 
approximate  40-percent  budget  reduction  in  total 
advertising  produced  enlistment  results  equal  to, 
and  occasionally  better  than,  other  geographic 
areas  unaffected  by  budget  reductions.  More¬ 
over,  in  explaining  the  enlistment  performance  of 
those  geographic  areas  subject  to  large  total 
budget  reductions,  differences  in  the  share  of  the 
total  budget  allocated  to  Joint  advertising  appear 
to  be  a  contributing  factor. 

Setting  budget  policy  based  solely  on  the 
empirical  findings  of  the  field  experiment, 
however,  does  not  take  into  account  changes  that 
may  have  occurred  in  enlistment  requirements,  in 
the  level  and  mix  of  other  recruiting  resources 
(e.g.,  bonuses,  pay,  recruiters)  or  in  changes  in 
overall  market  conditions  from  the  period  of  the 
experiment  (FY  1984)  to  the  present.  These 
factors  may  affect  the  level  of  future  advertising 
spending,  but  not  necessarily  the  proportion 
allocated  to  the  Joint  program.  The  research 
findings  suggest  that  the  pattern  of  growth  in 
Service-specific  advertising  during  the  past 
decade  need  not  continue  and  could  be  reversed 
without  adverse  impact  on  recruiting. 

REASONS  FOR  THE  FIELD 
EXPERIMENT 

Since  the  advent  of  the  all-volunteer  force, 
recruitment  advertising  spending  has  been  a 


source  of  controversy  among  the  Services,  the 
Office  of  the  Secretary  of  Defense,  and  the 
Congress.  While  no  one  disputes  the  fact  that 
advertising  remains  an  important  component  in 
the  recruiting  resource  mix,  differences  exist  over 
the  absolute  level  of  advertising  required  and  the 
shares  that  should  be  allocated  to  the  individual 
Services  and  the  Joint  Recruiting  Advertising 
Program  (JRAP).  In  FY  1981,  for  example,  the 
Congressional  Budget  Office  (CBO)  recom¬ 
mended  significant  increases  in  the  JRAP  with 
concurrent  reductions  in  Service-specific 
advertising.  This  CBO  study  sparked  a  dialogue 
between  the  Secretary  of  Defense  and  the  Military 
Services  about  the  level  and  mix  of  individual 
Service  and  Joint  advertising. 

During  these  discussions,  it  became  apparent 
that  additional  research  was  needed  and  that  an 
actual  field  experiment  represented  the  best 
approach.  A  field  experiment  was  chosen 
because,  while  logistically  complex  to  implement, 
it  overcame  the  difficulties  of  attempting  to  use 
historical  data  not  well  suited  to  forming  an 
empirical  basis  for  validating  present  or  proposed 
advertising  spending  levels. 

OBJECTIVE  OF  THE  FIELD 
EXPERIMENT 

Thus,  a  field  experiment  was  designed  and 
subsequently  implemented  to  generate  reliable, 
quantitative  data  on  the  contribution  of  advertising 
to  the  enlistment  process.  More  specifically,  the 
then-Assistant  Secretary  of  Defense  for 
Manpower,  Reserve  Affairs  and  Logistics,  Dr. 
Lawrence  J.  Korb,  stated  that  the  experiment  was 
expected  to  make  a  significant  contribution  in 
determining  the  optimum  level  and  mix  of 
Joint/Service-specific  recruitment  advertising  for 
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achieving  active,  enlisted,  non-prior  service 
recruiting  goals. 

The  results  of  the  experiment  bring  us  con¬ 
siderably  closer  to  answering  this  critical 
question.  Yet,  limitations  on  resources  and  restric¬ 
tions  on  the  design  of  the  field  experiment  do 
pose  legitimate  questions  as  to  how  far  one  can 
extend  the  findings  to  derive,  with  an  acceptable 
degree  of  confidence,  point  estimates  of  die 
optimum  level  and  mix  of  Joint  and  Service- 
specific  advertising.  However,  evidence  gathered 
from  the  experiment  does  show  that  a  reduced 
level  of  advertising  spending  would  not  adversely 
affect  recruiting  performance. 

DESIGN  OF  THE  FIELD  EXPERIMENT 

To  respond  to  the  stated  objective,  the 
Department  of  Defense  designed  a  field  experi¬ 
ment  The  original  concept  was  to  implement  a 
"full  factorial  design";  that  is,  one  in  which  many 
feasible  combinations  of  advertising  spending 
levels  and  mix  categories  were  tested.  In  essence, 
this  involved  test  cells  with  three  levels  of  Joint 
advertising  spending  and  three  levels  of  Service- 
specific  advertising  (nine  test  cells  total). 

As  shown  in  Figure  ES.l,  the  original  design 
considered  a  greater  range  of  budget  combinations 
than  were  eventually  executed.  The  original 
design,  had  it  been  implemented,  would  have 
provided  a  much  richer  database  and  would  have 
permitted  additional  statistical  analyses.  This,  in 
turn,  would  have  allowed  for  more  cross- 
validation  of  the  findings  presented  in  this  report. 

Because  of  the  difficulty  and  cost  involved  in 
implementing  such  a  design,  the  parties  com¬ 
promised  on  a  four-cell  design  that  included 
matching  low  and  medium  levels  of  Service 
expenditures  with  low,  medium  and  high  levels  of 
Joint  advertising  spending  as  depicted  on  Figure 


ES.2.  Although  not  a  complete  representation  of 
the  many  spending  combinations  represented  in 
the  original  design,  this  truncated  design 
permitted  testing  of  budget  mixes  relevant  to 
future  debates  about  advertising  spending.  From 
a  statistical  perspective,  however,  it  did  not  permit 
the  use  of  classical  two-way  analysis  of  variance 
to  isolate  the  factors  that  may  contribute  to 
differences  in  enlistment  response  (as  well  as 
other  measures)  across  the  test  cells.  As  an 
alternative,  the  empirical  findings  from  this  field 
experiment  were  derived  primarily  through  cross- 
sectional  regression  analysis.  This  statistical 
approach  for  the  truncated  design  proved  adequate 
to  develop  and  validate  the  empirical  findings  of 
the  field  experiment. 

The  geographical  units  comprising  the 
experimental  design  were  Areas  of  Dominant 
Influence  (ADIs)  developed  by  the  Arbitron 
Ratings  Company.  These  214  ADIs,  collectively 
covering  the  entire  United  States,  consist  of 
county  groups  reflecting  predominant  local 
television  viewing  patterns.  Cell  Blue  depicted  in 
Figure  ES.2,  for  instance,  consisted  of  fourteen 
ADIs,  comprising  about  eight  percent  of  the 
enlistable  population.  These  fourteen  ADIs  were 
geographically  dispersed  throughout  the  nation  to 
avoid  any  contamination  from  purely  regional 
effects.  The  ADI  markets  selected  for  each  test 
cell  were  statistically  matched  (balanced)  on  a 
number  of  variables,  including  size  of  population, 
enlistment  rates,  unemployment,  and  enlistment 
propensity,  dancing  the  ADI  markets  within  a 
test  cell  limited  the  confounding  influence  of  non¬ 
advertising  factors  in  subsequent  analyses. 
Lastly,  for  costs  and  administrative  consider¬ 
ations,  a  matched  subset  of  the  original  control 
cell  (White)  was  devised.  This  new  control  cell, 
consisting  of  31  ADIs,  was  identified  as  Cell 
Yellow. 
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Figure  ES.l  Proposed  Nine-Cell  Test  Design  Concept 


Figure  ES.2  Actual  Four-Cell  Design  Implemented  in  the  Field 


Joint  Advertising  Budget  Levels 


Percentages  represent  the  proportion  of  17-21-yeat-old  males  residing  in  the 
ADIs  covered  by  the  respective  cells.  Dollar  amounts  reflect  simulated  national 
spending  levels  in  each  cell  for  Service -specific  (numerator)  and  Joint 
(denominator)  advertising.  "Current'  spending  refers  to  the  amount  expended 
in  FY  1982,  the  baseline  budget  when  the  experiment  was  designed. 


/"v  vV'/V"  A"  l*.V«Va  'aJC 


EVALUATION  CRITERION  AND 
CHOICE  OF  MEASURES 

The  following  criterion  provides  the  basis  for 
evaluating  the  four  advertising  budget  policies 
tested:  The  recommended  advertising  policy  will 
be  one  providing  the  necessary  short-term  and 
long-term  contributions  to  the  attainment  of  the 
Services'  enlistment  requirements  at  the  least 
advertising  cost  to  the  Department  of  Defense. 
Thus,  the  measures  chosen  to  evaluate  the  field 
experiment  against  this  criterion  must  take  into 
account  both  near-  and  long-term  enlistment 
response  considerations. 

Evaluating  the  field  experiment  against  this 
criterion  involved  collecting  data  on  three  types  of 
measures: 

•  Observed  behavior  measures  as  output 
from  the  recruiting  systems.  These 
included  the  quantity  and  quality  of 
applicants  and  contracts  reported  by  the 
Military  Services. 

•  Reported  behavior  measures  relevant  to 
the  enlistment  process  as  derived  from  the 
Youth  Attitude  Tracking  Study.  These 
included  reported  contacts  with  recruiters 
and  reported  conversations  with  peers  and 
parents  about  the  military. 

•  Attitudinal  measures  derived  from  the 
Youth  Attitude  Tracking  Study.  These 
included  reported  intentions  to  enlist  in  the 
military,  as  well  as  the  strength  of  those 
intentions. 

IMPLEMENTATION  PLANS 

To  ensure  that  the  field  experiment  was 
properly  implemented,  media  plans  were 


developed  and  executed  by  the  Military  Services 
and  the  JRAP.  Comparison  and  tracking  of  actual 
media  execution  against  the  original  plan  revealed 
that  these  media  plans  conformed  to  the  budget 
guidelines  established  in  each  test  cell.  In 
addition,  actual  execution  of  the  media  plans 
paralleled  closely  the  plans  submitted  by  the 
Services  and  the  JRAP.  These  conformed  to  the 
design  criteria  for  the  field  experiment. 

DATA  COLLECTION 

The  staff  of  Wharton  Applied  Research  Center 
and  its  subcontractors,  with  extensive  coopera¬ 
tion  from  the  Department  of  Defense,  developed 
and  executed  a  comprehensive  data  collection 
process.  In  addition  to  the  media  information,  a 
variety  of  other  data  relevant  to  conditions  in  the 
marketplace,  such  as  numbers  of  recruiters, 
unemployment  rates  and  Service  mission/goals, 
was  collected. 

APPROACH  TOWARD  ANALYSES  OF 
THE  DATA 

Once  sufficient  data  were  collected  on  the 
measures  used  in  the  field  experiment,  several 
analytical  approaches  were  undertaken.  As  stated 
before,  because  of  the  truncated  nature  of  the  field 
experiment's  design,  conventional  two-way 
analysis  of  variance  and  covariance  could  not  be 
used.  Instead,  cross-sectional  statistical  regres¬ 
sion  techniques  and  various  standard  non- 
parametric  tests  of  significance  were  used. 

The  data  collected  for  the  72  geographic 
markets  (ADIs)  comprising  the  three  test  cells  and 
one  control  cell  on  the  observed  behavior 
measures  (i.e.,  applicants  and  enlistments)  were 
further  sub-divided  by  quality  and  Service. 
Quality  applicants  were  either  seniors  or  high 
school  diploma  graduates  who  performed  in  the 


top  half  of  the  aptitude  range  on  the  Armed  Forces 
Qualification  Test  (AFQT). 

For  each  subset  of  the  data,  a  series  of  cross- 
sectional  regression  analyses  was  perforated  in  an 
effort  to  determine  whether  the  differences  in 
budget  levels  (and  mix)  across  the  cells  correlated 
with  any  differences  in  the  observed  behavioral 
measures  across  the  same  cells.  More  than  100 
regressions  were  performed  on  subsets  of  the  data 
representing  the  observed  measures  of  behavior. 
Predictor  variables  included  not  only  advertising, 
but  also  unemployment,  race,  degree  of  urbani¬ 
zation,  number  of  recruiters,  and  a  dummy 
variable  for  the  appropriate  test  cell.  The  func¬ 
tional  forms  for  most  of  the  equations  used  were 
non-linear.  In  addition,  all  the  variables  were 
standardized  to  rates  using  a  population  variable. 
Regressions  were  also  run  by  pairing  cells 
together  and  using  predictor  variables  which 
reflected  changes  between  FY 1983  and 
FY  1984  (the  period  of  the  test).* 

Responses  on  the  Youth  Attitude  Tracking 
Study  regarding  reported  conversations  with 
parents  and  reported  contact  with  recruiters  were 
also  analyzed.  Data  on  these  measures  were 
examined  in  an  effort  to  determine  whether  a 
relationship  existed  between  reported  behaviors 
incidental  to  the  enlistment  process  and 
differences  across  the  cells  in  advertising 
spending.  The  responses  were  split  by  Service, 
with  changes  examined  between  the  test  period 
(FY  1984)  and  a  pre-test  period  (FY  1983).  For 
the  most  part,  a  two-tailed  "t"  test  was  used  to 
determine  whether  statistically  significant 
differences  occurred  in  various  subsets  of 
response  data  between  and  among  the  cells. 

Lastly,  various  responses  on  the  Youth 
Attitude  Tracking  Study  intended  as  measures  of 
likelihood  or  propensity  to  enlist  in  the  military 
were  examined.  Prior  research  suggests  that 
these  data,  purported  to  measure  propensity  to 


enlist,  do  in  fact  represent  lagged  indicators  of 
future  enlistment  behavin'.  Parametric  and  non- 
parametric  statistical  tests  were  performed  on 
these  data  in  a  manner  similar  to  those  described 
above  for  measures  of  reported  behavior  (i.e., 
conversations  with  parents  and  contact  with 
recruiters). 

These  three  sets  of  measures  were  analyzed  to 
cross-validate  the  findings  from  one  set  of 
measures  using  another  set.  Of  concern  was 
the  fact  that  analyzing  only  data  representing 
measures  that  reflected  primarily  near-term  results 
(applicants  and  enlistments)  would  overlook 
equally  important  long-term  effects  of  advertising. 
Hence,  the  measures  on  purported  behaviors  and 
expressions  of  interest  in  military  service  were 
also  included.  Response  data  on  these  measures, 
which  typically  lead  enlistment  results,  were 
analyzed  to  determine  if  they  were  affected  by 
changes  in  advertising  spending. 


♦The  models  and  estimation  methodology  used  in 
this  study  are  consistent  with  those  of  previous 
research  on  factors  affecting  enlistment  supply. 
However,  more  recent  efforts  suggest  that 
recruiter  behavior  variables  might  be  important  in 
manpower  supply  models  (Dertouzos,  1985; 
Carroll,  Lee  ana  Rao,  1986).  Systematic 
changes  in  recruiter  behavior  can  alter  the  quan¬ 
tity  and  quality  of  enlistments  and  can  make 
estimating  the  impact  of  recruiting  resources, 
including  advertising,  difficult.  To  the  extent 
that  changes  in  recruiter  behavior  are  correlated 
with  changes  in  advertising  expenditures,  the 
magnitude  of  the  advertising  effect  may  be  under¬ 
estimated.  Accounting  for  such  factors  simul¬ 
taneously  for  all  four  Services  is  a  demanding 
task  well  beyond  the  scope  of  this  study. 
Accordingly,  any  effects  that  recruiter  behavior 
variables  might  have  had  on  the  findings  of  this 
experiment  are  unknown. 
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EMPIRICAL  FINDINGS  FROM  THE 
ANALYSES 

The  findings  reported  in  this  section  emerged 
from  the  analyses  of  the  data  on  the  three  types  of 
measures  used  to  evaluate  the  field  experiment 
Conclusions  drawn  from  these  findings  as  well  as 
possible  budget  implications  are  described  in 
a  subsequent  section  of  this  Executive  Summary. 

Observed  Measures  of  Behavior 

With  respect  to  subsets  of  data  on  enlistments, 
applicants  and  the  rate  of  applicant  conversion  to 
contracts,  the  analyses  revealed  that: 

•  The  test  cell  with  the  lowest  total  adver¬ 
tising  spending  (Cell  Blue)  produced 
applicant  and  enlistment  results  equal  to, 
and  occasionally  better  than,  the  control 
cell. 

•  Other  test  cells  with  considerably  larger 
total  spending  than  Cell  Blue  (but  still 
somewhat  less  than  the  control  cell) 
produced  results  no  better  than,  and 
occasionally  poorer  than,  the  control  cell 
(Cell  Yellow). 

•  None  of  the  test  cells  provided  results  in 
terms  of  enlistment  or  applicant  share  by 
Service  that  differed  to  a  statistically 
significant  degree  from  the  shares 
observed  in  the  control  cell. 

•  In  the  first  year  of  significant  changes  to 
advertising  spending  levels,  it  appears  that 
the  contribution  of  advertising  to  recruit¬ 
ing  system  performance  is  either  quite 
small  or  virtually  non-existent. 


Reported  Measures  of  Behavior 

With  respect  to  data  on  reported  measures, 
such  as  conversations  with  parents  or  contact  with 
recruiters,  the  analyses  revealed  the  following: 

•  The  reported  incidence  of  conversations 
with  parents  about  enlistment  did  not 
differ  between  and  among  the  test  cells 
and  the  control  cell  to  a  statistically 
significant  degree. 

•  The  reported  incidence  of  recruiter  contact 
occurred  to  a  greater  degree  (also  statisti¬ 
cally  significant)  in  die  test  cell  with  the 
lowest  total  spending  (Cell  Blue). 

•  When  responses  were  analyzed  by 
reference  to  a  specific  Service,  the 
findings  noted  in  (1)  and  (2)  above  did  not 
differ  for  references  to  the  Army  but  did 
differ  for  the  other  three  Services. 
References  to  the  Air  Force,  for  example, 
declined  in  the  test  cells  to  a  greater  extent 
than  in  the  control  cell.  This  decline  also 
occurred  in  one  test  cell  for  the  Navy  but 
not  for  the  Marine  Corps. 

Reported  Measures  of  Attitudes 

Because  prior  research  suggests  that  expres¬ 
sions  of  interest  in  eventual  military  service  by  the 
prospect  audience  appear  to  lead  trends  in  actual 
enlistment  results,  these  data  were  examined  as 
part  of  the  field  experiment  Three  separate 
variables  purporting  to  measure  interest  in  military 
service  as  derived  from  the  Youth  Attitude 
Tracking  Study  were  employed.  These  are 
referred  to  generally  as:  (1)  unaided  mentions  of 
joining  the  military;  (2)  after  interviewer 
prompting,  an  expression  of  likelihood  of  joining 


the  military;  and  (3)  after  interviewer  prompting, 
a  variable  representing  a  composite  of  favorable 
responses  to  interest  in  joining  any  one  or  more  of 
the  individual  Services.  The  variable  used  in  each 
case  represented  a  proportion  of  respondents  who 
stated  that  they  would  "definitely"  or  "probably" 
join  the  military. 

The  findings  that  resulted  from  analyses  of  the 
attitudinal  data  were  inconclusive.  While 
differences  did  occur  to  a  statistically  significant 
degree  between  and  among  test  cells  and  the 
control  cell,  no  clear  pattern  emerged.  Any 
attempt  to  cross-validate  the  findings  of  one 
approach  with  another  only  served  to  reinforce  the 
inconsistency  and  inconclusiveness  of  the  results. 
Therefore,  expressions  of  interest  in  military 
service,  while  possibly  a  valid  predictor  of  future 
enlistment  response,  do  not  appear  to  be 
correlated  in  any  meaningful  or  predictable 
manner  with  changes  in  advertising  spending  as 
reflected  in  the  test  cells.  This  is  not  to  say  that 
advertising  bears  no  relationship  to  attitudes 
toward  military  service,  but  only  that  the  field 
experiment  did  not  reveal  any  consistent  pattern 
permiting  conclusions  to  be  drawn  from  the 
results. 


BUDGET  IMPLICATIONS  OF  THE 
FINDINGS 


As  noted  before,  the  field  experiment 
employed  different  advertising  spending  levels 
under  conditions  as  they  existed  in  FY  1984. 
Assuming  recruiting  resources  and  policies  and 
overall  market  conditions  have  not  changed 
dramatically  during  the  intervening  years,  results 
of  the  field  experiment  suggest  that  a  lower  level 
of  total  advertising  spending  could  be  adopted 
nationally  than  that  proposed  for  FY  1987.  More 
specifically,  the  proposed  FY  1987  total 
advertising  budget  could  be  reduced  by  17*25 


percent  (In  fact  Congress  reduced  that  budget 
request  by  18  percent) 

While  an  estimate  for  total  advertising 
spending  in  FY  1987  can  be  inferred  from  the 
spending  level  used  in  Cell  Blue  (after  appropriate 
adjustments  for  inflation  and  adjustments  for  the 
non-media  portion  of  the  budget),  a  mathematical 
model  was  subsequently  developed  to  simulate 
plausible  combinations  of  Joint  and  total  Service- 
specific  advertising  at  different  total  spending 
levels.  Basically,  the  model  showed  that  the  mix 
of  Joint  and  total  Service-specific  advertising 
employed  in  Cell  Blue  (at  a  national  level  in 
FY  1984  of  $15  million  for  Service-specific  and 
$16  million  for  Joint)  was  actually  sub-optimal. 
That  is,  a  somewhat  different  mix  of  Joint  and 
total  Service-specific  advertising  budgets  would 
have  produced  better  results  for  the  same  total 
spending.  This  model  assumes  that  an  interaction 
with  respect  to  recruiting  system  performance 
exists  between  Joint  and  total  Service-specific 
advertising  spending.  Likewise,  the  total  budget 
in  FY  1987  is  both  too  large  (based  on  the 
empirical  findings  from  the  experiment)  and  sub- 
optimal  in  its  mix  (based  on  the  modeling 
employed). 


RECOMMENDATIONS 


The  empirical  findings  of  the  field  experiment 
suggest  that  the  Department  of  Defense  can  reduce 
its  total  advertising  spending  without  adversely 
affecting  recruiting  performance.  However,  pre¬ 
cise  spending  levels  cannot  be  determined  solely 
from  the  field  experiment.  The  data  do,  however, 
show  the  most  cost-effective  direction  for  future 
spending  on  advertising  and  provide  approxima¬ 
tions  of  the  size  of  short-term  budget  adjustments. 

The  size  of  the  Joint  advertising  budget 
should  be  increased  as  Service-specific  budgets 
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CHAPTER  1 
INTRODUCTION 


BACKGROUND 

During  the  past  several  years,  Congress  has 
increasingly  been  concerned  about  the  advertising 
costs  incurred  for  the  recruitment  of  military 
personnel.  Historically,  inquiries  have  been  made 
by  the  Secretary  of  Defense  concerning  the  level 
of  such  advertising,  its  appropriate  allocation 
between  the  Joint  and  Service-specific  programs 
and  its  effectiveness.  Unfortunately,  insufficient 
data  were  available  to  address  adequately  these 
concerns. 

In  1981,  the  Congressional  Budget  Office 
(CBO)  recommended  significant  increases  in  the 
Joint  Recruiting  Advertising  Program  (JRAP) 
with  concurrent  reductions  in  Service-specific 
advertising  to  produce  an  overall  savings  in 
advertising  expenditures.  The  Secretary  of 
Defense  agreed  with  CBO's  concept  and 
proposed  to  double  the  size  of  the  Joint  Service 
Program  in  1982  while  reducing  Service-specific 
advertising.  (See  Appendix  A  for  additional 
details1.)  Such  cutbacks  were  viewed  by  the 
individual  Services  as  detrimental  to  their  ability 
to  meet  recruitment  goals  and  to  sustain  the  gains 
made  in  the  quality  of  recruits.  Joint  advertising 
was  viewed  as  "corporate"  or  umbrella  adver¬ 
tising  complementing  the  main  advertising  thrust 
by  the  individual  Services.  The  Services  con¬ 
tended  that  the  projected  increase  in  Joint 
advertising  could  not  adequately  compensate  for 
the  reduction  in  total  Service-specific  advertising. 

In  the  summer  of  1981,  the  Secretary  of 
Defense  continued  to  review  the  issues  concern¬ 
ing  the  proper  Service-specific/Joint  advertising 


mix  and  the  most  efficient  total  advertising  level. 
Because  of  the  lack  of  research  in  these  areas,  the 
Secretary  decided  to  conduct  a  major  field  test.  In 
the  meantime,  the  advertising  mix  and  total 
funding  remained  at  the  Fiscal  Year  (FY)  1981 
levels.  The  test  was  conducted  throughout  FY 
1984  (October  1983  -  September  1984).  Data 
were  collected,  validated  and  collated  during  FY 
198S.  The  analysis  began  in  FY  1986  and  was 
completed  in  FY  1987. 

NEED  FOR  THE  TEST 

As  indicated,  insufficient  data  were  available 
to  provide  direction  concerning  the  appropriate 
level  of  DoD  advertising  and  the  specific  mix  of 
Joint  and  Service-specific  spending.  Indeed, 
historical  data  on  DoD  advertising  were  difficult 
to  analyze  for  the  following  reasons: 

•  Historical  data  did  not  provide  indepen¬ 
dence  of  variables.  The  amount  of  adver¬ 
tising  and  number  of  recruiters  frequently 
varied  simultaneously.  As  a  result,  it  was 
difficult  to  disentangle  their  independent 
effects.  Further,  it  is  often  unclear 
whether  advertising  affects  recruitment 
(sales)  or  if  the  level  of  recruitment  (sales) 
affects  the  level  of  advertising.  Both  pos¬ 
itions  have  historically  been  argued  (e.g., 

•Memorandum  dated  8  July  1983  from  Lawrence 
J.  Korb,  Assistant  Secretary  of  Defense  (Man¬ 
power,  Reserve  Affairs  &  Logistics)  to  the 
Assistant  Secretaries  of  the  Army,  Navy,  and  Air 
Force. 


"we  need  more  advertising  to  maintain 
high  sales"  or  "in  markets  with  low  sales, 
we  need  advertising  to  increase  sales”). 

•  Historical  data  did  not  provide  sufficient 
variance.  Advertising  expenditures  fre¬ 
quently  vary  in  only  a  narrow  range.  The 
prevalent  budgeting  practice  keeps  the 
Services  and  the  Joint  shares  of  the  total 
budget  relatively  constant  from  year  to 
year.  Hence,  evaluation  of  alternative 
policies  which  differ  significantly  from 
past  practice  requires  extrapolation  beyond 
the  range  of  observed  data. 

•  Historical  data  are  under  no  uniform 
system  of  measurement.  Recruiting  sys¬ 
tem  data,  such  as  number  of  recruiters  and 
quotas  levels,  are  often  not  available  in  the 
same  unit  (geography,  time,  etc.)  of 
observation  across  Services.  This  makes 
analysis  and  comparisons  difficult  The 
effect  of  advertising  on  the  enlistment  pro¬ 
cess  may  be  small  when  compared  to  the 
effect  of  other  variables  such  as  recruiters 
or  unemployment  Omitted  variables  or 
profound  measurement  errors  in  historical 
data  can  seriously  bias  results  or  com¬ 
pletely  obscure  the  effect  of  advertising. 

For  these  reasons  and  for  the  lack  of 
conclusive  research  in  the  area  of  industry  and 
brand  advertising  and  their  respective  applications 
to  military  recruiting,  the  Secretary  of  Defense 
established  this  advertising  experiment 

OBJECTIVE  OF  THE  TEST 

The  DoD  Advertising  Mix  Test  was  initiated 
because  "the  Department  of  Defense  does  not 
have  a  methodology  relating  and  quantifying  the 


effect  of  varying  levels  of  advertising  to  actual 
enlistments."2  The  objective  was  to  respond  to 
the  following  question:  "What  is  the  optimum 
mix  of  Joint/Service-specific  recruitment  advertis¬ 
ing  for  achieving  active,  enlisted,  non-prior 
service  (NPS)  goals  at  different  levels  of  total 
DoD  recruitment  advertising?"3  The  experiment 
was  designed  as  an  in-market  test  which  would 
generate  the  necessary  quantitative  data. 

The  aim  of  the  test  was  to  capture  and 
quantify  the  impact  of  different  budget  policies  on 
the  recruiting  system's  performance.  These 
issues,  pertaining  specifically  to  active,  enlisted, 
non-prior  service  advertising  were: 

•  What  is  the  impact  of  the  size  of  the 
advertising  budget  on  recruiting  system 
performance? 

•  What  is  the  impact  of  different  proportions 
of  Service-specific  and  Joint  advertising 
budgets  on  recruiting  system  perfor¬ 
mance? 

•  Does  an  effective  mix  between  Service- 
specific  and  Joint  advertising  budgets 
depend  on  the  size  of  the  overall 
advertising  budget? 

Since  only  one  actual  national  budget  policy 
could  be  implemented  in  a  given  year,  these 
budget  policy  issues  were  addressed  by  using  per 
capita  advertising  expenditure  levels.  These  were 
based  on  systematically  different  budgets  in  each 
of  four  sets  of  television  markets.  The  perfor¬ 
mance  of  the  recruiting  system  was  assessed 
through  both  short-  and  long-term  measures.  In 

2Korb,  op.cit. 

3Korb,  op.cit. 


the  short  term,  the  contribution  of  advertising  to 
the  recruitment  effort  was  measured  by  its  effect 
on  meeting  accession  missions,  contract  objec¬ 
tives  and  quality  goals.  In  the  longer  term,  these 
budgets  were  to  be  reviewed  with  regard  to  their 
contribution  to  maintaining  favorable  attitudes 
toward  the  Military  Services. 

REPORT  OVERVIEW 


direct  implications  are  explained  in  Chapter  6. 
Chapter  7  presents  additional  analyses  of  the 
experimental  data  and  discusses  the  implications 
of  the  results.  Finally,  a  summary  of  the 
experiment  and  recommendations  is  presented  in 
Chapter  8. 

ISSUES  NOT  ADDRESSED 


This  report  describes  the  design,  implemen¬ 
tation,  results  and  implications  of  the  DoD 
Advertising  Mix  Test  This  study  was  a 
collaborative  research  initiative  spanning  a  four- 
year  period.  It  focused  on  the  effectiveness  and 
efficiency  of  military  recruiting  advertising.  The 
research  centered  on  a  one-year,  controlled 
experiment  conducted  in  sets  of  matched 
television  markets  from  October  1,  1983  until 
September  30, 1984.  The  total  level  of  recruiting 
advertising  expenditures  and  the  mix  of 
expenditures  between  Service-specific  and  Joint 
advertising  were  systematically  varied  in  the 
experiment  Sharply  different  advertising  budget 
policies  were  implemented  (on  a  pro-rata  basis)  in 
each  of  four  different  sets  of  markets.  Recruiting 
system  performance  measures  were  established 
and  computed  across  the  four  sets  of  markets  to 
assess  the  comparative  effects  of  the  different 


The  DoD  Advertising  Mix  Test  was  NOT 
designed  to  measure  or  evaluate  the  following 
issues: 

The  effectiveness  of  the  media  mix, 
Le.,  the  combination  of  media  utilized  (e.g., 
television,  radio,  magazines,  direct  mail, 
etc.).  This  test  allocated  budget  levels  to  each 
cell  for  Service-specific  and  Joint  advertising. 
The  Services  and  the  Joint  program  director 
then  independently  decided  how  to  allocate 
those  budgets  across  various  media.  Thus, 
this  test  cannot  evaluate  whether  the 
advertising  resources  would  have  been  better 
spent  in  radio,  for  instance,  rather  than  in  TV. 
This  test  also  cannot  categorically  determine 
whether  the  results  obtained  in  one  market 
were  a  function  of  a  more  effective  media  mix 
than  that  used  in  another  market 


advertising  budget  policies  on  shorter  and  longer 
term  recruiting  system  performance. 

The  report  is  organized  as  follows:  Chapter  2 
presents  the  research  perspective,  the  design  of 
the  experiment  and  the  actual  advertising  expendi¬ 
ture  levels  achieved  in  the  year  of  the  experiment. 
Chapter  3  discusses  the  development  of  a  criterion 
for  evaluating  the  experiment.  The  measures  used 
to  assess  recruiting  system  performance  are 
described  in  Chapter  4.  Chapter  3  reviews  the 
approach  taken  in  analyzing  die  experiment  and 
discusses  related  methodological  and  statistical 
issues.  The  results  of  the  experiment  and  their 


The  appropriateness  of  the  share  of 
total  advertising  budget  allocated  to 
each  individual  Service.  In  designing  the 
test,  each  Service  received  the  same 
proportion  of  the  total  Service-specific  budget 
that  it  had  received  in  the  recent  past  Thus, 
this  test  cannot  address  whether  the  results 
would  have  changed  if,  for  example,  the  Air 
Force  had  received  a  larger  share  of  the 
Service-specific  budget  Nor  can  the  test 
results  resolve  the  appropriateness  of  the 
budget  levels  historically  allocated  to  each  of 
the  Services. 


The  effectiveness  of  the  creative 
content  of  the  various  advertising 
messages.  During  the  test  period,  the 
Services  and  the  Joint  program  maintained  the 
then  current  thematic  and  creative  content  of 
the  advertising  campaigns.  Thus,  this  test 
|  cannot  comment  on  whether  the  results  would 
[  have  been  different  if  the  creative  content  had 
changed  or  if  the  results  observed  were 
predominantly  a  function  of  the  differences  in 
creative  content 

|  PROJECT  TEAM 

In  May  19S3,  on  behalf  of  the  Office  of  the 
Assistant  Secretary  of  Defense  (Manpower, 
t  Reserve  Affairs  and  Logistics),  the  Office  of 

!  Naval  Research  awarded  a  contract  to  perform  the 

DoD  Advertising  Mix  Test  to  the  Wharton 
Applied  Research  Center  of  the  University  of 
Pennsylvania.  Because  of  the  massive  scope  and 
complexity  of  the  test  a  project  team  from  a 
consortium  of  contractors,  consultants  and 
advisory  personnel  was  assembled.  Each  of  the 
five  major  contractor  groups  had  special  areas  of 
expertise  essential  to  the  completion  of  the 
project  These  contractors  were: 

Wharton  Applied  Research  Cotter 
(WARC,  now  the  Wharton  Colter  for 
Applied  Research),  as  the  primary 
contractor,  was  responsible  for  the  entire 
scope  of  the  project  This  included  partici¬ 
pation  in  the  development  of  the  final 
experimental  design,  identification  of  data  to 
be  collected  and  development  of  systems  for 
its  collection.  With  assistance  from  the 
RAND  Corporation,  WARC  also  selected  and 
assigned  Areas  of  Dominant  Influence  (ADIs- 
the  geographic  unit  used  to  define  television 
markets)  into  their  respective  test  and  control 
cells,  developed  the  evaluation  criterion, 


identified  the  measures  of  effectiveness, 
selected  the  research  methods  to  analyze  the 
test,  and  reported  the  test  findings.  In 
addition,  WARC  documented  and  maintained 
the  data  base. 

CACI,  Inc.-Federal  coordinated  the 
complex  management  tasks  involved  in  the 
preparation  of  the  DoD  Advertising  Mix  Test 
CACI  provided  the  administrative  and  facility 
support  for  all  meetings  and  briefings.  They 
prepared  the  graphic  artwork,  typing  and 
report  production  support  for  all  contract 
deliverables. 

PEP  Systems,  Inc.  collected  from  the 
Services  and  JRAP  historical  and  experi¬ 
mental  national  advertising  data.  PEP 
produced  post-buy  analyses  of  these  adver¬ 
tising  data  and  prepared  a  database  containing 
total  national  advertising  expenditures  and 
impressions  by  market  (ADI),  by  advertiser, 
by  month  and  by  media  type.  PEP  Systems 
constructed  the  database  from  detailed  data 
submissions  by  each  advertiser  covering  each 
separate  national  advertising  media  purchase. 

Arthur  D.  Little,  Inc.  (ADL)  developed 
two  interview  protocols  administered  to 
industry  associations  and  their  advertising 
agencies  to  ascertain  the  use  and  benefits  of 
industry  advertising. 

OR/M  S  Dialogue,  Inc.  (now  Rao 
Associates)  prepared  a  report  on  the 
conceptual  models  to  evaluate  military  enlist¬ 
ment  advertising  effectiveness  and  provided  a 
report  reviewing  the  empirical  studies  of 
enlistment  response  to  advertising.  They 
conducted  a  series  of  independent  analyses  of 
the  experimental  data  and  participated  in  the 
development  of  the  final  report. 


CHAPTER  2 

RESEARCH  DESIGN  AND  IMPLEMENTATION 

As  discussed  in  the  prior  chapter,  the  laboratory  while  military  advertising  researchers 


objective  of  this  study  was  to  relate  various  DoD 
advertising  budget  levels  to  actual  enlistment 
behavior.  In  doing  so,  the  study  aimed  to  identify 
the  appropriate  mix  of  Joint/Service-specific 
recruitment  advertising  at  different  levels  of  total 
DoD  expenditures.  This  chapter  reviews  the 
research  perspective  from  which  this  objective 
was  investigated  and  the  constraints  which 
conditioned  the  study's  design.  The  planning  and 
fielding  of  the  experiment  as  well  as  the  allocation 
of  advertising  expenditures  are  described.  In 
addition,  the  chapter  includes  an  analysis  of  the 
general  performance  of  the  enlistment  process 
during  the  period  of  the  test 

RESEARCH  PERSPECTIVE 

The  military  manpower  recruiting  system  is  a 
large  and  complex  one.  Recruitment  performance 
depends  strongly  on  broad  economic  and  social 
conditions.  It  is  also  influenced  by  factors  such 
as  military  pay  and  bonus  levels,  recruiter  efforts, 
etc.  However,  previous  studies  would  seem  to 
indicate  that  advertising  is  not  a  major  determinant 
of  performance.  Overall,  the  recruiting  system  is 
a  dynamic  one,  changing  as  requirements,  policy, 
and  the  environment  change,  and  as  multiple 
decision  makers  compete  and  cooperate  within  a 
fairly  rigid  resource  allocation  process.  In  this 
environment,  controlled  advertising  experimenta¬ 
tion  and  data  analysis  are  unlikely  to  yield  the 
crisp,  clean  results  that  are  obtained  in  the 
physical  or  social  sciences.  Major  sources  of 
variation  can  be  largely  controlled  in  the 


clearly  cannot  control  unemployment.  Econo¬ 
metric  analyses  and  other  methods  for  evaluating 
historical  data  suffer  even  more  from  these  system 
complexities.  In  this  experiment,  system  com¬ 
plexities  have  been  addressed  by  establishing  tight 
and  uniform  controls.  This  was  accomplished  by 
matching  treatment  cells  for  historical  perfor¬ 
mance  and  by  repeating  advertising  treatments  in 
multiple  markets. 

The  research  challenge  was  to  produce 
managerially  useful  guidelines  to  evaluate  the 
level  and  mix  of  advertising  for  the  Department  of 
Defense.  These  guidelines  should  be  revised  and 
enriched  as  the  system  evolves  over  time.  The 
study  was  approached  in  the  following  ways: 

•  By  using  multiple  measures  of  per¬ 
formance:  observed  behavior  such  as 
contracts  and  applications,  reported 
behavior  such  as  recruiter  contacts  and 
conversations  with  parents,  and  attitudinal 
measures  such  as  intentions  to  join  the 
armed  services. 

•  By  placing  less  dependence  on  specific 
individual  analyses  and  more  emphasis  on 
consistency  among  a  variety  of  analyses 
using  the  above  multiple  measures. 

•  By  searching  for  the  direction  and  ranges 
of  expenditures  which  can  be  modified 
over  time  rather  than  by  seeking  precise 
and  static  decision  rules. 
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DESIGN  CONSTRAINTS 


To  assess  the  relationship  between  advertising 
budgets  and  enlistment  system  performance  in  an 
ideal  world  would  involve  several  sequential 
steps.  Criteria  for  evaluating  the  impact  of 
advertising  budgets  on  recruitment  would  first  be 
developed.  Then,  measures  for  evaluating  the 
criteria  would  be  created,  validated,  tested  for 
reliability  and  sensitivity,  and  finally  selected. 
Next,  an  experiment,  incorporating  different 
budget  combinations,  would  be  designed.  The 
test  would  allow  each  budget  level  to  be 
implemented  many  tunes.  Relevant  data  protocols 
and  definitions  would  be  developed.  Then,  each 
test  budget  would  be  implemented  simultaneously 
on  a  nationwide  basis  for  a  prolonged  period. 
Data  would  be  collected,  validated  and  collated 
and  the  experiment  analyzed.  Finally,  recommen¬ 
dations  based  on  the  analyses  would  be  made. 

Clearly,  practical  realities  such  as  time,  budget 
and  personnel  limitations  conspire  against  this 
ideal.  As  a  result,  the  definitive  assessment  of  all 
possible  budget  combinations  and  policy  options 
cannot  be  undertaken.  However,  improvements 
in  budgeting  policy  can  be  achieved  by  the 
systematic  assessment  of  a  subset  of  practical 
alternatives  to  the  current  budget  policy.  In 
addition,  prudent  modifications  to  the  idealized 
research  process  can  be  made  to  accomplish  the 
objective  within  a  managerially  useful  time 
frame. 

As  a  result  of  these  constraints,  the  criteria- 
and  measures-development  phases  were  carried 
out  simultaneously.  At  the  same  time,  data  were 
being  collected  in  the  field.  The  tasks  could  not 
have  been  undertaken  in  this  way  if  the  members 
of  the  research  team  had  not  had  previous 
experience  in  the  field  of  recruitment  advertising. 
This  experience  enabled  the  ream  to  make 
reasonable  judgments  about  the  data  specification 


as  well  as  the  projected  outcomes  of  the  criteria 
and  measures. 

Many  test  budgets  could  not  be  implemented 
simultaneously  on  a  nationwide  basis  for 
prolonged  periods.  As  a  result,  the  experimental 
budget  levels  were  carried  out  on  a  pro-rata  basis 
in  matched  sets  of  television  markets  over  a  one- 
year  time  frame.  This  was  possible  because  each 
military  advertiser  involved  in  the  test  was 
required  to  provide  a  systematic  set  of  imple¬ 
mentation  plans  (discussed  later  in  this  chapter). 
Another  modification  to  the  "ideal"  study  dealt 
with  limiting  the  number  of  budget  policies  to  be 
examined.  As  stated  in  the  following  paragraphs, 
ultimately  four  budget  levels  were  tested. 

Number  of  Test  Cells 

Design  consultants  -  Canter,  Achenbaum  and 
Associates  -  initially  developed  a  nine-cell,  two- 
factorial  design  which  incorporated  low,  current 
ami  high  levels  of  both  Joint  and  Service-specific 
advertising  (Figure  2.1).  However,  the  Office  of 
the  Assistant  Secretary  of  Defense  (Manpower, 
Reserve  Affairs  and  Logistics)  [OASD 
(MRA&L)]  recognized  in  the  early  stages  of  this 
effort  the  importance  of  involving  the  Services 
and  the  Joint  Recruitment  Advertising  Program  in 
the  development  of  the  research  design.  Through 
the  sponsorship  of  OASD  (MRA&L),  the 
Wharton  Center  then  held  a  series  of  meetings 
with  Joint  and  Service  representatives  to  discuss 
and  revise  the  design  of  the  experiment. 

Spending  Levels  in  Each  Cell 

Because  of  the  difficulty  and  cost  involved  in 
implementing  such  an  experiment,  the  parties 
compromised  on  a  four-cell  design.  It  included 
matching  current  and  lower  levels  of  Service 


Service-Specific  Advertising  Service-Specific  Advertising 

Budget  Levels  Budget  Levels 


Figure  2.1 

Proposed  Nine-Cell  Test  Design  Concept 


Joint  Advertising  Budget  Levels 


Note:  Caster,  Achenbanm  and  Associates  proposed  the  above  nine-cell  design. 

component!  o l  which  were  actually  implemented  in  the  field  aa  depicted  in 
the  following  Figure  13.  Cell*  containing  diagonal  line>  were  not  pat  of 
the  actual  teat  implemented  in  the  field. 


Figure  2.2 

Actual  Four-Cell  Design  Implemented  in  the  Field 


Joint  Advertising  Budget  Levels 


spending  level!  in  each  cell  for  Service-specific  (numerator)  and  Joint 
(denominaor)  advertising.  "Current”  spending  refers  to  the  amount  expended 
in  FY  1982,  the  baseline  budget  when  the  experiment  was  designed. 


expenditures  with  low,  current  and  high  levels  of 
Joint  spending  as  shown  in  Figure  2.2.  While 
not  a  complete  representation  of  all  the  spending 
combinations  proposed  in  the  original  design,  this 
truncated  design  does  permit  testing  of  budget 
mixes  relevant  to  future  advertising  budget  levels. 

As  noted  in  Figure  2.2,  Cell  White  was 
designated  the  control  cell,  maintaining  FY  1982 
levels  of  both  Joint  ($16  million)  and  Service- 
specific  ($68  million)  advertising.  Later,  Cell 
Yellow,  a  subset  of  Cell  White,  was  designated  as 
the  control  cell.  The  existence  of  such  a  cell  was 
the  result  of  budget  cuts  which  the  Navy  and  the 
Joint  program  sustained  during  the  period  of  the 
advertising  experiment.  This  required  a  reduction 
in  the  size  of  the  control  cell.  As  a  result,  the 
burden  of  field  data  collection  was  reduced.  Cell 
Blue  had  the  same  FY  1982  level  of  Joint 
spending  ($16  million)  but  a  lower  Service- 
specific  ($15  million)  budget  Cell  Green  had 
lower  levels  of  Joint  advertising  ($4  million)  and 
the  FY  1982  levels  of  Service-specific  ($68 
million)  spending.  And  finally.  Cell  Red  had 
much  higher  levels  of  Joint  spending  ($40 
million)  and  much  lower  levels  of  Service-specific 
advertising  ($15  million)  than  the  control  cell. 
Once  the  total  Service-specific  budget  levels  were 
established  for  each  cell,  individual  Service 
budgets  were  established.  Those  were  based  on 
the  proportion  historically  achieved  by  each 
Service  in  the  allocation  process. 

ADI  Matching  Process 

The  Area  of  Dominant  Influence  (ADI),  a 
television  market,  was  the  unit  of  analysis  for  this 
test  A  county-based  geographic  unit  commonly 
used  in  advertising  analysis,  an  ADI  is  a  group  of 
one  or  more  counties,  the  plurality  of  whose 


households  receive  electronic  media  from  one 
common  location.  Taken  together,  ADIs  uniquely 
account  for  each  county  in  the  continental  United 
States.  Specifically,  the  1981-82  Arbitron  ADIs 
were  used  as  the  basis  for  this  test.  Thus,  Cell 
White  consisted  of  ADIs  which  included  76 
percent  of  the  country's  young  men  between  the 
ages  of  17  and  21  inclusive.  The  three  other  cells 
each  comprised  8  percent  of  that  age  group.  Cell 
Yellow  (a  subset  of  the  control  cell)  consisted  of 
16  percent  of  the  country.  Cell  Yellow  sub- 
sequendy  became  the  strict  control  cell  as  a  result 
of  budget  cuts.  These  budget  cut  adjustments 
occurred  in  those  Cell  White  ADIs  not  included  in 
the  Cell  Yellow  subset.  The  remaining  Cell  White 
ADIs  were  considered  unacceptable  for  analysis 
because  the  spending  level  in  these  ADIs  fell 
below  FY  1982  budget  levels. 

Clearly,  individual  ADIs  differed  from  one 
another  on  many  characteristics  which  could 
impact  recruiting  system  performance.  These 
sources  of  performance  variation  needed  to  be 
controlled  or  accounted  for  to  assess  accurately 
the  impact  of  advertising  on  the  recruiting  system. 
Two  steps  were  taken  to  do  this.  First,  data  on 
major  known  sources  of  variation  (such  as 
differences  in  unemployment  rates,  levels  of 
recruiter  effort,  civilian  income  levels,  etc.)  were 
collected  for  each  ADI  and  were  explicidy 
included  as  co-variates  in  the  analysis  of  the 
experiment.  These  efforts  are  described  in 
Chapters  5  and  6  of  this  report.  Second,  the  four 
advertising  cells  in  the  experiment  were  closely 
matched  to  each  other  on  a  variety  of  these 
characteristics.  The  objective  was  to  develop  four 
cells  which  were  as  much  alike  as  possible  so  that 
differences  in  recruiting  system  performance 
could  be  direedy  attributed  to  differences  in  the 
advertising  induced  during  the  experiment. 


The  RAND  Corporation  provided  technical 
assistance  in  assigning  markets  (ADIs)  to  test 
cells.  The  primary  market-matching  criteria 
included  the  market's  previous  enlistment  rate  and 
the  market  share  for  each  Service  within  the  ADI. 
The  secondary  matching  criteria  consisted  of  the 
level  of  unemployment,  the  percentage  of  non¬ 
whites  and  the  propensity  of  youth  to  join  the 
military.  The  ADIs  represented  in  each  test  cell 
exhibited  a  fairly  uniform  geographic  distribution 
and  appeared  roughly  balanced  according  to  ADI 
market  sizes.  Various  cell  configurations  (groups 
of  ADIs)  best  conforming  to  these  matching 
criteria  were  developed  and  reviewed  by  the 
Services  and  JRAP.  Finally,  one  configuration 
was  selected.  The  matched  sets  of  ADIs  were 
then  randomly  assigned  by  Wharton  to  be  Cells 
Blue,  Red,  Green,  and  White  (Control). 
Descriptions  regarding  the  matching  criteria  of  the 
test  ADIs  and  cells  are  provided  in  Appendix  B. 

DEVELOPING  IMPLEMENTATION 
PLANS  FOR  EACH  TEST  CELL 

Once  all  parties  agreed  with  the  research 
design,  the  Wharton  Center  requested  each 
Service  and  JRAP  to  submit  advertising,  media 
and  translation  plans  for  each  of  the  budget  levels. 
The  purpose  of  these  plans  was  to  assure  that  the 
experimental  budget  levels  were  consistently 
implemented  on  a  pro-rata  basis  in  the  test  ADIs. 
In  addition,  the  plans  guaranteed  that  all  decisions 
concerning  the  uses  of  the  budget  funds  were 
made  by  the  advertisers.  This  information 
explained  in  detail  how  each,  in  collaboration  with 
its  advertising  agency,  planned  to  spend  the 
advertising  funds  at  each  budget  level.  There 
were  three  steps  to  this  process. 

The  advertising  plans  reflected  the  nationwide 
effect  of  each  cell's  budget.  They  specified,  for 


each  budget  level,  how  funds  would  be  allocated 
to  local  versus  national  advertising.  They  pro¬ 
vided  information  regarding  production,  agency 
and  market  research  costs.  Lastly,  the  plans 
included  data  about  the  number  of  markets  which 
were  to  receive  advertising  support  for  each 
budget  level.  Appendix  C  discusses  in  detail  the 
planned  advertising  expenditures. 

The  media  plans  provided  further  details. 
These  plans  specified  the  allocation  of  national 
funds  across  markets  (ADIs)  and  across  media 
(television,  radio,  print,  direct  mail).  In  addition, 
the  media  plans  provided  advertising  schedules. 

Translation  plans  then  converted  the  national 
and  local  media  plans  into  actual  schedules  of 
planned  advertising  for  each  test  cell  and  test 
market.  These  market-by-market,  planned  adver¬ 
tising  schedules  formed  the  basis  for  advertising 
purchasing  actions  during  the  test  year. 

To  achieve  the  desired  pro-rata  advertising 
levels  in  each  market,  a  number  of  quite  detailed 
decisions  and  implementation  steps  were  required 
to  purchase  advertising  time,  space,  and  material. 
Purchasing  actions  were  based  on  projected 
advertising  exposures  rather  than  strictly  on  costs 
because  of  the  price  structure  of  advertising  and 
its  availability.  The  translation  plans  then  spelled 
out  other  detailed  steps  to  be  taken  including:  (1) 
national  buys  -which  media  would  be  bought  on  a 
network  or  national  basis;  (2)  cut  outs  (the 
deletion  of  advertising  messages,  usually  in 
electronic  media)  -  which  national  (network)  buys 
would  be  cut  out  in  which  markets;  (3)  spot  or 
market  buys  (buy-ups)  -  which  additional  media 
were  needed  in  which  markets;  (4)  confirmation 
and  control  mechanisms  -  for  both  buy-ups  and 
cut-outs;  (5)  make-good  procedures  -  how  would 
preemptions  be  made  good  or  compensated  for, 
and  (6)  cost  of  execution  -  media  costs,  media 
savings,  and  cut-out  costs  (additional  television 
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network  charges  for  deletion  of  an  announcement 
in  selected  ADIs). 

Representatives  from  the  Joint  and  the  four 
Service  advertising  programs  compiled  these 
plans  into  "factbooks”  which  Wharton  and  OASD 
(MRA&L)  reviewed  Wharton's  principal  investi¬ 
gator  and  the  project  manager  then  met  to  discuss 
these  plans  with  JRAP,  each  of  the  Services,  and 


representatives  of  their  respective  advertising 
agencies.  These  meetings  succeeded  in  resolving 
remaining  difficulties  involved  in  planning  for  the 
implementation  of  the  test 

The  advertising  implementation  plans  pro¬ 
duced  different  advertising  and  media  configura¬ 
tions  in  the  various  tests  cells.  For  example, 
national  television  advertising  was  not  an  effective 


Table  2.1 

Planned  vs.  Actual  Advertising  Deliveries 
(Dollar  Expenditures  per  17-21- Year-Old  Male) 


JOINT 

PLANNED 

SERVICES 

TOTAL 

JOINT 

ACTUAL 

SERVICES 

TOTAL 

National 

$1.48 

$4.34 

CELL  WHITE 

$5.82 

$1.61 

$4.32 

$5.93 

Loral 

- 

$1.17 

$1.17 

- 

$0.87 

$0.87 

Total 

$1.48 

$5.51 

$6.99 

$1.61 

$5.19 

$6.80 

National 

$1.55 

$0.79 

CELL  BLUE 

$2.34 

$1.63 

$1.80 

$3.43 

Local 

- 

$0.55 

$0.55 

- 

$0.53 

$0.53 

Total 

$1.55 

$1.34 

$2.89 

$1.63 

$2.33 

$3.96 

National 

$0.39 

$4.49 

CELL  GREEN 

$4.88 

$0.41 

$4.07 

$4.48 

Local 

- 

$1.16 

$1.16 

- 

$0.89 

$0.89 

Total 

$0.39 

$5.65 

$6.04 

$0.41 

$4.96 

$5.37 

National 

$3.91 

$0.83 

CELL  RED 

$4.74 

$3.92 

$2.21 

$6.13 

Local 

- 

$0.51 

$0.51 

- 

$0.64 

$0.64 

Total 

$3.91 

$1.34 

$5.25 

$3.92 

$2.85 

$6.77 

National  -  Media  is  purchased  on  a  national  basis.  The  advertising  agency  is  executing  the  media  plan. 

Local  -  Media  is  purchased  on  an  individual  market  basis.  The  execution  is  made  by  the  local 
advertising  agency  representative  and/or  the  market's  commanding  officer. 


alternative  for  some  advertisers  at  low  budget 
levels.  In  general,  the  Services  with  low  budget 
levels  tended  to  spend  proportionally  more  on 
local  and  print  advertising.  These  decisions  were 
made  by  the  advertisers.  The  DoD  Advertising 
Mix  Test  was  not  designed  to  assess  directly  the 
effect  of  these  media-mix  decisions.  Rather,  the 
test  takes  these  media  decisions  as  outcomes  of 
the  prevailing  practices  in  the  military  advertising 
system.  It  is  possible,  of  course,  that  different 
media-mix  choices  would  have  resulted  in 
different  test  outcomes. 

PLANNED  VERSUS  ACTUAL  ADVER¬ 
TISING  DELIVERED 

Table  2.1  presents  the  working  media 
expenditures  planned  and  actually  delivered  for 
each  test  cell  during  the  study.  It  is  expressed  on 
a  per  capita  basis.  This  is  calculated  by  dividing 
the  advertising  and  recruiter  variables  by  the  pop¬ 
ulation  of  17-21-year-old  males  in  that  ADI. 

Considering  the  difficulties  of  implementation 
and  the  vagaries  of  media  delivery,  the  total 
deliveries  by  cell  are  remarkably  close  to  the 
planned  deliveries.  In  the  cells  receiving  the  1982 
level  of  Service-specific  advertising  (Yellow  and 
Green),  the  per  capita  expenditures  are  roughly 
equivalent.  Similarly,  the  Blue  and  Red  cells  had 
low  levels  of  Service-specific  advertising. 
Overall,  the  advertising  deliveries  also  achieved 
the  desired  levels  of  variance.  Figure  2.3  shows 
a  plot  of  Joint  versus  Service-specific  spending 
by  ADI.  As  can  be  seen,  only  two  markets 
received  inappropriate  levels  of  advertising.  One 
Red  cell  market  (Grand  Junction,  Colorado) 
received  much  more  advertising  than  planned  for 
any  cell.  As  a  result,  it  was  eliminated  from  the 
analysis.  Another  market  in  Blue  cell  (Harrisburg- 
York,  Pennsylvania)  had  advertising  levels 
similar  to  Yellow  cells.  Consequently,  it  was 
analyzed  as  part  of  the  Yellow  cell  group. 


Data  Collection 

The  full  significance  of  the  close  adherence  of 
actual  to  planned  delivery  levels  can  only  be 
appreciated  when  one  considers  the  magnitude  of 
the  data  collection  effort  The  data  collection 
involved  coordination  among  OSD,  JRAP,  the 
advertising  directors,  recruiter  management 
personnel  and  accession  policy  personnel  of  the 
four  Services,  in  addition  to  five  advertising 
agencies,  five  direct  mail  fulfillment  houses*,  the 
Defense  Manpower  Data  Center  (DMDC)  and 
three  project  subcontractors. 

In  all,  the  data  collection  involved  508 
separate  project  input  submissions  including  the 
following:  recruiter  data  (number  of  recruiters, 
location,  er^.);  local  and  national  advertising  data; 
contract  and  accession  goal  information;  appli¬ 
cants,  contracts  and  individuals  in  DEP  (Delayed 
Entry  Program)  by  quality  level;  gross  and 
qualified  national  leads  information;  and  a  variety 
of  exogenous  variables  such  as  unemployment 
data,  racial  composition,  urbanization,  and 
household  income.  (Some  data,  such  as  that  for 
the  DEP,  were  collected  only  for  historical 
purposes  and  were  not  a  factor  in  the  final 
analysis.)  A  copy  of  the  data  collection  plan  is 
included  in  Appendix  D.  The  data  collection  was 
a  major  undertaking  both  in  size  and  scope.  The 
Services  and  JRAP  are  commended  for  the  level 
of  cooperation  and  collaboration  they 
demonstrated  throughout  the  implementation  and 
data  collection  phases  of  this  test. 


’Direct  mail  fulfillment  houses  are  those  firms 
which  prepare  and  mail  various  types  of  adver¬ 
tising  material  to  households  using  selected  lists. 
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Figure  2.3 

Advertising  Delivery  by  Market 


From  these  data  submissions,  the  Wharton 
Center  assembled  a  monthly  database  covering  the 
period  from  FY  1981  through  FY  1984  (October 

1980  through  September  1984).  These  were 
converted,  where  necessary,  from  main  station* 
to  ADI  observations.  The  conversion  program 
was  developed  by  the  Wharton  staff  and  was 
based  on  the  number  of  17-  21-year-old  males  in 
the  specific  counties  across  the  country.  This 
conversion  program  is  documented  in  a  user's 
manual.  The  FY  1983  and  FY  1984  data  used  in 
the  analysis  are  complete.  However,  in  the  FY 

1981  and  FY  1982  time  period,  some  of  the  data 
items  were  either  missing  or  not  available.  The 
Wharton  Center  has  included  only  what  was 
received.  Other  researchers  should  take  note  that 
some  data  elements  are  missing  in  earlier  years 
across  some  of  the  Services. 

The  benefits  of  this  comprehensive  data 
collection  effort  extend  beyond  its  original 
intention.  The  Services  and  JRAP  have  continued 
to  collect  and  report  recruiting  and  advertising 
data  on  a  formal  basis  in  a  standardized  format 
The  data  are  maintained  by  DMDC  and  provide  an 
invaluable  database  for  use  in  advertising  plan 
analysis  and  budget  justification. 

TEST  ENVIRONMENT 

The  performance  of  the  military  recruiting 
system  during  the  year  of  the  test  FY  1984, 


declined  compared  to  the  previous  year. 
Nationally,  the  economy  improved,  unemploy¬ 
ment  declined,  and  military  enlistment  contracts 
for  all  categories  of  recruits  also  dropped. 
Broader  measures  of  recruiting  system  perfor¬ 
mance  went  down  as  well.  There  were  declines 
in  the  size  of  the  delayed  entry  pools  of  the 
Services  and  the  number  of  applicants  tested.  In 
addition,  smaller  proportions  of  youths  indicated 
a  positive  propensity  to  join  the  military.  They 
also  reported  fewer  contacts  with  military 
recruiters  and  a  reduction  in  the  number  of 
conversations  with  their  parents  about  enlistment 
in  the  military. 

These  decreases  in  recruiting  system  produc¬ 
tivity  during  the  year  of  the  test  occurred  in  all 
cells  including  cells  White  and  Yellow  where  no 
changes  in  advertising  budget  policy  were  tested. 
As  a  result,  it  is  most  reasonable  to  infer  that  the 
decreases  in  recruiting  system  performance  were 
not  caused  by  the  advertising  changes  but  were 
the  result  of  the  overall  economic  conditions  in  the 
United  States  during  the  year  of  the  test  These 
environmental  fluctuations  require  that  the  study's 
findings  be  evaluated  by  comparing  recruiting 
system  performance  in  markets  where  advertising 
was  changed  with  cell  Yellow  (control)  where 
advertising  was  not  changed.  This  is  the  purpose 
of  a  control  cell.  The  approach  to  and  results  of 
these  analyses  are  described  in  Chapter  5  and  6  of 
this  report 


’Main  Station  -  Army  Recruiting  Battalion,  Navy 
Recruiting  District  Marine  Corps  Recruiting 
Station  or  Air  Force  Recruiting  Squadron. 


CHAPTER  3 


A  CRITERION  FOR  EVALUATING  ADVERTISING  POLICIES 


The  objective,  scope,  design  and  implementation 
of  the  DoD  Advertising  Mix  Test  have  now  been 
described.  This  chapter  reviews  the  criterion  by 
which  the  study  was  evaluated.  To  develop  this 
criterion,  the  Wharton  Applied  Research  Center 
(WARC)  undertook  research  in  three  areas. 
These  were: 

•  A  literature  review  of  advertising  and 
marketing  research  relevant  to  evaluating 
advertising  effectiveness. 

•  Primary  exploratory  research  in  the  form 
of  semi-structured  interviews,  conducted 
to  identify  and  categorize  prevailing 
motivations  for  collaborative  advertising. 

•  A  study  of  mathematical  models  pertaining 
to  the  effects  of  advertising  on  military 
enlistments. 

Although  the  following  pages  will  provide  an 
overview  of  the  findings,  further  detailed 
discussion  of  these  results  is  provided  in  the 
report  produced  in  September  1984  entitled  A 


(Appendix  E). 

FINDINGS  OF  THE  LITERATURE 
REVIEW 

Although  several  studies  have  focused  on 
generic  or  commodity  advertising  for  agricultural 


projects,  the  overwhelming  majority  of  advertis¬ 
ing  research  has  been  conducted  at  the  corporate 
or  brand  level.  This  research  has  dealt  with  two 
areas.  The  first  attempts  to  relate  aggregate 
phenomena,  such  as  corporate  or  brand  sales,  to 
advertising  resources.  The  second  investigates 
the  impact  of  advertising  resources  on  consumer 
attributes  such  as  advertising  awareness,  copy 
recall,  and  intention  to  buy. 

Although  there  is  no  clear  consensus  as  to  the 
role  of  consumer  attribute  variables,  there  is  some 
significant  evidence  that  stated  purchase  intention 
measures  are  valid  predictors  of  purchase 
behavior.  The  literature  review  reinforces  the 
requirement  that  any  intermediate  variables 
employed  in  evaluating  the  test  must  be  rigorously 
tested  for  both  validity  and  reliability. 

No  guidance  is  provided  in  the  literature 
concerning  the  simultaneous  evaluation  of 
industry  advertising  and  firm-specific  advertising. 
Several  studies  have  estimated  the  effect  of 
advertising  on  sales  for  a  number  of  corporations 
in  the  same  industry.  However,  results  of  these 
investigations  have  differed  sharply.  In  addition, 
none  considered  a  collaborative  or  jointly 
sponsored  advertising  campaign.  Other  studies 
have  estimated  the  effects  of  advertising  on  sales 
for  generic  or  commodity  advertising  campaigns 
but  have  not  simultaneously  considered  firm- 
specific  campaigns. 

A  distinction  between  national,  corporate 
advertising  and  cooperative  (largely  trade  channel) 
industry  advertising  was  identified.  In  a  recent 
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book  on  this  topic,  Young  and  Grcyser  (1983)  cannot  be  applied  directly  to  evaluating  military 
assert  that:  recruiting  advertising  policies. 


There  exists  very  little  in  the  way  of  formal 
evaluation  methods  for  assessing  cooperative 
advertising  effectiveness.  The  usual  advertis¬ 
ing  tracking  services  and  evaluation  services 
offer  no  systematic  way  of  helping  managers 
assess  co-op's  effectiveness4. 

Previous  studies  of  military  recruiting 
advertising  are  of  limited  usefulness.  The  focus 
of  the  effort  to  understand  the  effects  of  military 
recruiting  advertising  has  been  primarily  at  the 
individual  Service  level.  A  substantial  number  of 
enlistment  supply  models  have  been  developed 
since  the  inception  of  the  All  Volunteer  Force  in 
1973.  However,  very  few  of  these  studies 
estimate  the  effects  of  advertising  on  enlistments. 
Those  models  which  do  make  these  estimates 
disagree  about  the  underlying  model  structure  and 
about  the  effects  of  advertising  on  enlistments. 
Despite  the  lack  of  consensus  about  the  magnitude 
of  advertising  effects,  the  literature  on  military 
advertising  indicates  that  marketing  variables  do 
have  significant  impact  The  fielding  of  a 
controlled  experiment  such  as  the  DOD 
Advertising  Mix  Test  is  advocated  in  this 
literature. 

The  underlying  themes  in  the  evaluation 
literature  are  sales  effectiveness  and  economic 
efficiency.  In  the  private  sector,  these  competing 
factors  are  incorporated  into  profit  maximization 
models.  These  models  set  advertising  budgets  so 
that  the  incremental  contribution  from  the  last  sale 
just  equals  the  advertising  cost  of  achieving 
the  sale. 

Unfortunately,  a  mechanism  for  measuring 
recruiting  "sales"  in  terms  of  dollars  is  lacking. 
As  a  result,  the  concept  of  profit  maximization 


FINDINGS  OF  INDUSTRY  INTER¬ 
VIEWS 

Collaborative  advertising  occurs  when  a 
group  of  manufacturers  or  suppliers  of  a  similar 
product  or  service  join  together  to  advertise  in  a 
generic  way  their  product,  service  or  industry. 
Although  collaborative  advertising  is  not  rare  in 
the  marketplace,  the  literature  dealing  with  its 
evaluation  offers  little  guidance  for  evaluating  the 
DoD  Advertising  Mix  Test  To  gain  a  better 
understanding,  the  WARC  project  staff  inter¬ 
viewed  20  industry  trade  associations.  The 
associations  were  selected  using  the  following 
guidelines:  (1)  multi-million  dollar  advertising 
budgets;  (2)  a  strong  representation  by  industries 
providing  financial  services  (because  of  their 
comparability  with  each  other);  (3)  industries 
competing  with  other  industries  for  market  share 
(e.g.,  the  beef  or  pork  industry);  and  (4)  a  diverse 
mixture  of  industries. 

Of  the  20  candidate  organizations,  one  was 
excluded  from  the  sample  because  its  advertising 
campaign  was  not  collaboratively  funded.  The 
remaining  19  organizations  were: 

Florida  Department  of  Citrus 

National  Pork  Producers  Council  American 

American  Dairy  Association 

Florida  Fruit  and  Vegetable  Association 

National  Live  Stock  and  Meat  Board 

American  Florists  Marketing  Council 


4Young,  R.F.  and  S.A.  Grcyser.  1983. 
Managing  Cooperative  Advertising:  A  Strategic 
Approach.  Lexington,  Massachusetts. 


Quality  Bakers  of  America 
Investment  Company  Institute 
Communication  Workers  of  America 
U.S.  Committee  for  Energy  Awareness 
The  American  Bankers  Association 
American  Council  of  Life  Insurance 
International  Ladies'  Garment  Workers  Union 
Savings  and  Loan  Foundation,  Inc. 
Independent  Insurance  Agents  of  America 
American  Sheep  Products  Council,  Inc. 
American  Gas  Association 
American  Heart  Association 
Linen  Supply  Association  of  America 


Directors  of  Advertising,  Directors  of 
Marketing,  or  Executive  Directors  of  the  organi¬ 
zations,  as  deemed  appropriate  by  the  responding 
organization,  were  interviewed.  In  one  case,  the 
advertising  agency  was  viewed  as  most  know¬ 
ledgeable  for  responding  to  interview  questions. 


Five  major  objectives  for  collabora¬ 
tive  advertising  were  identified: 


•  To  achieve  economies  of  scale  - 
This  was  the  most  common  reason  given. 
As  long  as  the  industry  members  share  a 
common  message,  an  association  can 
purchase  a  larger,  more  professional 
advertising  campaign  than  its  individual 
members.  Production  of  a  series  of  adver¬ 
tisements  is  less  expensive  per  unit  than 
production  of  individual  ones.  Media 
costs  become  more  efficient  because  of 
volume  discounts  and  greater  negotiating 
power.  In  addition,  economies  can  be 
realized  when  advertising  is  placed  on  a 
national  vs.  a  regional  or  local  basis. 


•  To  address  secondary  target 
groups  -  In  some  cases  industry 


members  advertise  collectively  because  as 
a  group  they  can  reach  audiences  that  will 
influence  the  behavior  of  the  primary 
target  group.  In  these  instances,  collabora¬ 
tive  advertising  is  a  more  effective  means 
of  indirect  advertising. 


To  address  the  primary  group  from 
a  new  perspective  -  A  number  of 
industry  groups  perceive  collaborative 
advertising  as  being  more  credible  than 
individual  advertising.  In  many  cases,  the 
goal  of  collaborative  advertising  was 
described  as  the  "creation  of  an  appro¬ 
priate  image”  which  allows  the  member 
firms  to  take  a  different  posture  when 
advertising. 


To  eliminate  "free  riders"  -  The  "free 
rider"  problem  arises  in  a  non-branded, 
commodity  industry  where  there  is  little 
product  differentiation.  Advertising  by 
one  producer  inadvertently  benefits  its 
competitors.  Collaborative  advertising 
eliminates  this  problem  since  all  industry 
members  contribute  to  the  advertising 
campaign. 


To  stimulate  goodwill  -  Collaborative 
advertising  was  reported  as  contributing  to 
goodwill  within  an  industry.  Seeing  posi¬ 
tive  advertising  makes  industry  members 
"feel  good"  about  the  industry  and 
encourages  members  to  improve  their 
efforts  and  industry  support 


These  objectives  fall  naturally  into  the 
efficiency/effectiveness  dichotomy  familiar  from 
the  literature  review.  They  translate  almost 
directly  into  hypotheses  about  the  roles  of  Joint 
and  Service-specific  advertising. 
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Economies  of  scale  may  support  the  use  of 
Joint  advertising.  This  is  based  on  the  extent  to 
which  potential  recruits  view  similarities  between 
the  various  Services.  Joint  advertising  may  allow 
savings  in  production  and  media  costs  when  all 
purchases  are  made  through  one  agency  or 
channel.  However,  the  perceived  requirement  for 
Service  differentiation  may  not  be  satisfied  by 
generic  Joint  advertising.  Therefore,  a  potential 
recruit's  differential  response  to  Joint  and  Service- 
specific  advertising  must  be  measured  to 
determine  if  economies  of  scale  and  effectiveness 
of  Joint  advertising  outweigh  the  effectiveness  of 
Service-specific  advertising. 

Reaching  secondary  target  groups  with  the 
same  message  may  be  hypothesized  to  be  done 
more  effectively  with  Joint  advertising  than  with 
Service-specific  advertising.  A  Joint  campaign, 
designed  to  enhance  positive  attitudes  about 
military  service  by  giving  the  common  benefits  of 
all  the  Services,  may  more  readily  appeal  to  the 
general  population.  Parents,  friends  and  guidance 
counselors  may  be  motivated  to  encourage 
members  of  the  primary  target  group  to  consider 
contacting  a  military  recruiter.  This  indirect 
advertising  may  also  encourage  enlistment  of  non¬ 
target  group  individuals.  As  a  result.  Joint 
advertising  may  be  hypothesized  to  encourage 
more  enlistments  from  older  or  non-primary- 
individuals. 

In  the  context  of  the  Department  of  Defense,  it 
can  be  hypothesized  that  Joint  advertising  reaches 
the  primary  target  group  with  a  more  generic 
message  than  Service-specific  advertising. 
Within  certain  segments  of  the  youth  population, 
this  broader  message  for  the  Military  Services 
may  be  perceived  as  having  greater  credibility. 
As  a  result,  it  may  be  seen  as  more  effective  in 
enhancing  the  image  of  the  military  and  in 
improving  basic  attitudes  toward  enlistment 


Joint  advertising  may  prevent  a  free-rider 
problem  which  may  be  hypothesized  to  exist 
when  one  Service  conducts  a  generic  advertising 
campaign  which  could  benefit  other  Services. 
Collaborative  (Joint)  advertising  removes  the  need 
to  reallocate  budgets  among  the  Services.  How¬ 
ever,  it  is  not  clear  whether  a  free-rider  problem 
can  arise  at  the  Department  of  Defense.  Each  of 
the  four  competitive  Services  essentially  repre¬ 
sents  a  "branded"  Service.  If  the  advertising 
compaigns  are  sufficiently  different  from  one 
another,  a  free-rider  problem  will  not  occur  when 
one  Service  uses  a  generic  advertising  message. 
Spillover  effects  may  be  insufficient  to  require 
Joint  advertising. 

Instead,  Joint  advertising  may  control  the 
extent  to  which  the  Services  can  differentiate 
themselves  from  one  another.  If  only  Service- 
specific  advertising  existed,  the  Services  could 
concentrate  their  advertising  on  gaining  market 
share  rather  than  expanding  the  market  by 
improving  basic  attitudes  toward  enlistment  The 
Services  may  differentiate  themselves  to  such  an 
extent  as  to  be  dysfunctional.  One  can  hypothe¬ 
size  that  less  Service  differentiation  occurs  with 
more  Joint  advertising. 

Joint  advertising  may  have  substantial 
goodwill  effect  for  the  Department  of  Defense.  It 
may  be  hypothesized  that  Joint  advertising  helps 
generate  a  sense  of  pride  in  the  military  service 
both  among  service  personnel  and  civilians.  This 
goodwill  may  help  stimulate  enlistments  and 
reenlistments  among  the  active  and  reserve 
components  of  the  military. 

FINDINGS  OF  CONCEPTUAL  MODELS 

Four  conceptual  models  were  developed  to  aid 
in  understanding  the  contribution  of  advertising  to 
the  enlistment  process.  Two  of  these  were 


Service  quotas  and  quality  standards 
promote  "brand  switching"  by  potential 
applicants  among  the  Services. 


reported  by  Ambar  S.  Rao  in  A  Model  for  Joint 
and  Service  Advertising  Budgets  (October  1983) 
and  two  bv  Hau  L.  Lee  in  A  Generalized  Model 
for  Joint  and  Individual  Service  Advertising  on 
Enlistment  for  the  Armed  Forces  (November 


1983).  All  four  models  are  discussed  in  detail  in 


A  Criterion  for  Evaluating  Ad 


It  Policies 


(September  1984)  (Appendix  E).  The  models 
hypothesize  two  kinds  of  contributions  from 
military  recruitment  advertising: 


•  Direct  contributions  to  die  signing  of 
enlistment  contracts  (closing  the  sale). 


•  The  importance  of  the  enlistment  decision 
encourages  discussions  with  influencers, 
and  these  influencers  may  therefore 
constitute  an  important  advertising  target 

•  The  primary  target  group  is  a  small 
(narrow)  segment  of  the  population  and  a 
highly  transitory  group  not  easily  reached 
by  existing  media. 


•  Indirect  contributions  to  improving 
altitudes  toward  military  service  among 
youth  who  are  approaching  or  in  the  prime 
enlistaUe  age  group,  and  among  the 
individuals  who  may  influence  them. 

The  conceptual  models  minimize  die  sum  of 
Service-specific  and  Joint  advertising  expendi¬ 
tures  (efficiency)  while  assuring  sufficient  adver¬ 
tising  expenditures  for  each  Service  to  meet  its 
enlistment  contract  requirements  (effectiveness). 

In  addition  to  the  recurring  themes  of 
efficiency  and  effectiveness,  several  unique 
elements  of  the  recruiting  environment  w ere 
highlighted  in  the  formulation  and  evaluation  of 
these  conceptual  models: 

•  Repeat  purchases,  an  important  aspect  of 
product  sales,  is  a  negligible  feature  of  the 
enlistment  process. 

•  A  stricdy  sequential  buying  cycle  occurs: 
exposure  to  the  possibility  of  military 
service,  recruiterfs)  contact,  testing,  and 
signing  of  enlistment  agreements. 


•  An  intermediate  stated  intention  to  enlist 
can  be  hypothesized  to  precede  actual 
enlistment  and  serve  as  a  useful  and  valid 
predictor  for  subsequent  behavior. 

CONCLUSION 

The  literature  review,  the  survey  of  industry 
advertisers  and  the  conceptual  model  development 
support  the  use  of  both  efficiency  and 
effectiveness  criteria  for  evaluating  the  DoD 
Advertising  Mix  Test 

Aggregate  sales  response  (or  responses  of 
intermediate  variables  which  accurately  predict 
sales)  was  found  to  be  an  appropriate  base  for  the 
effectiveness  criterion.  Short-term  contributions 
of  advertising  include  meeting  accession  and  con¬ 
tract  missions  and  maintaining  quality  standards. 
Longer-term  enlistment  contributions  of  advertis¬ 
ing  include  maintaining  favorable  attitudes  toward 
the  Military  Services.  A nother  long-term  contribu¬ 
tion  concerns  promoting  the  consideration  of 
military  service  among  high-quality  potential 
candidates. 


An  economic  efficiency  criterion  was  also 
found  to  be  useful  in  evaluating  advertising 
effectiveness.  In  the  private  sector,  economic 
efficiency  is  typically  conceptualized  as  profit 
maximization.  Because  recruiting  "sales"  cannot 
be  translated  into  dollars,  profit  maximization  is 
replaced  by  cost  minimization  as  an  economic 
efficiency  criterion  for  evaluating  the  test 
In  the  private  sector,  a  firm  will  encourage 
and  accept  All  additional  sales  generated  by 
advertising.  The  only  limitation  is  the  amount  of 
product  that  can  physically  and  economically  be 
produced  to  satisfy  the  new  demand.  In  contrast 
the  military  is  limited,  by  quotas,  from  accepting 
All  enlistment  contracts  (sales)  potentially  gener¬ 
ated  as  a  result  of  advertising.  In  consequence. 


the  goal  of  advertising  for  the  military  is  to 
maintain  military  strength  (sales)  at  the  lowest 
advertising  cost  possible.  In  addition,  this  adver¬ 
tising  is  to  develop  and  maintain  a  favorable 
attitude  among  potential  candidates  and  the  people 
who  could  influence  them. 

Consequently,  the  following  criterion  will  be 
used  to  evaluate  the  advertising  budget  policies 
being  tested  in  the  DoD  Advertising  Mix  Test: 

Evaluation  Criterion:  The  recommended 
advertising  policy  will  be  one  providing  the 
necessary  short-tom  and  long-term  contri¬ 
butions  to  the  attainment  of  the  Services' 
enlistment  requirements  at  minimum  adver¬ 
tising  cost  to  the  Department  of  Defense. 
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CHAPTER  4 


EVALUATION  MEASURES 


The  previous  chapter  reviewed  the  overall 
criterion  chosen  to  evaluate  the  various  DoD 
advertising  budget  strategies  implemented  in  the 
experiment  This  chapter  takes  the  next  logical 
step  in  the  research  process  by  describing  the 
selection  and  validation  procedures  used  in 
choosing  a  set  of  evaluation  measures.  Chapter  5 
will  cover  the  analytical  approaches  that  employed 
these  measures  to  evaluate  the  data  collected  from 
the  field  experiment 


THE  SET  OF  MEASURES  CHOSEN 


The  set  of  measures  selected  to  assess  the 
contribution  of  various  advertising  budget 
strategies  implemented  in  the  experiment  includes 
aggregate  enlistment  response  measures 
(observed  behavior)  and  aggregate  intermediate 
measures  (reported  behavior  and  attitudinal  data). 
The  hypothesis  is  that  a  change  in  advertising 
(total  dollars  and  mix  of  Service-specific  and 
Joint)  will  have  a  noticeable  effect  on  the 
following  measures: 


Observed  Behavior  Measures 


Quantity  and  quality  of  enlistment  con¬ 
tracts  by  Service 

Quantity  and  quality  of  applicants  taking 
die  qualifying  test  by  Service 
Conversion  ratios  of  applicants  into 


contracts 


Reported  Behavior  Measures 


•  Reported  recruiter  contact 

•  Reported  conversations  about  the  military 


m 


Attitudinal  Measures 


Reported  intentions  to  enlist  in  the  military 
Reported  strength  of  intentions 


APPROACH  TOWARD  SELECTION  OF 
THE  MEASURES 


The  choice  of  measures  used  to  evaluate  the 
field  experiment  depends  on  four  factors:  first, 
the  degree  of  consistency  between  the 
experiment's  overall  evaluation  criterion  and  the 
chosen  measures;  second,  the  extent  to  which  the 
measures  adequately  capture  the  potential  effects 
of  advertising  along  each  stage  of  the  enlistment 
decision  process;  third,  the  availability  of  data  sets 
which  adequately  serve  as  the  chosen  measures; 
and  fourth,  whether  the  measures  themselves 
conform  to  acceptable  standards  of  validity  and 
reliability.  The  next  sections  review  each  of  those 
four  factors  in  turn. 


Consistency  Between  the  Evaluation 
Criterion  and  Chosen  Measures 


The  overall  criterion  for  the  field  experiment 
entailed  an  evaluation  as  to  how  well  each  tested 
advertising  strategy  provided  the  necessary  short- 
and  long-term  contribution  to  the  Services' 
enlistment  requirements  at  the  least  cost  to  the 
Department  of  Defense.  Thus,  the  set  of  mea¬ 
sures  chosen  must  collectively  support  an  analysis 
that  spans  a  sufficiently  long  planning  horizon  for 
the  respective  advertising  strategies  to  be 
evaluated  under  this  criterion. 

Under  ideal  circumstances,  using  a  single 
measure  to  evaluate  the  field  experiment  would  be 


ismm 


the  clearest  and  simplest  approach.  For  example, 
the  marketing  literature  is  replete  with  studies  of 
purchase  situations  that  use  sales  (in  DoD  terms, 
enlistment  contracts)  as  the  single  best  outcome 
measure. 

However,  two  difficulties  exist  in  military 
recruiting  that  argue  against  the  use  of  a  single 
measure.  First,  self-imposed  enlistment  quotas 
may  bias  this  measure  of  advertising's  effect  by 
obscuring  its  true  underlying  contribution. 
Second,  given  the  length  of  the  enlistment 
decision  (purchase)  cycle,  a  measure  of 
advertising's  effect  on  only  current  enlistments 
may  overlook  important  leading  indicators  of  its 
longer-term  impact  on  enlistments. 

Measures  that  Span  the  Enlistment  Deci¬ 
sion  Process 

Because  the  criterion  for  evaluating  the 
experiment  demanded  that  the  longer-term 
consequences  of  changed  advertising  policies  be 
taken  into  account,  a  set  of  measures  was  chosen 
that  collectively  spanned  the  entire  range  of  the 
enlistment  decision  process. 

Figure  4.1  portrays  a  schematic  representation 


of  the  enlistment  process  and  the  associated  data 
sources  for  each  stage  in  that  process.  Thus,  the 
observed  and  reported  behavior  measures  and  the 
attitudinal  measures  reflect  conditions  at  each 
stage  throughout  the  enlistment  decision  process. 

Data  Sets  as  Sources  for  the  Measures 

Data  already  collected  by  the  Department  of 
Defense  facilitated  the  development  of  these 
measures  that  reflect  the  changes  at  each  stage  of 
the  enlistment  process.  For  example,  the  annual 
DoD-sponsored  Youth  Attitude  Tracking  Study 
(YATS)  provides  a  useful  comparison  over  time 
of  youth  attitudes  toward  military  service. 

The  YATS  also  served  as  a  source  for  the 
measures  on  reported  youth  behaviors,  such  as 
contact  with  recruiters  or  discussions  with 
parents.  The  Defense  Manpower  Data  Center 
(DMDC)  provided  data  on  subsequent  stages  in 
the  enlistment  decision  process.  Specifically, 
DMDC  provided  statistics  on  potential  applicants 
for  military  service  and  subsequent  counts  of 
actual  contracts  signed  to  enter  military  service. 

These  sets  of  measures  are  described  in  more 
detail  in  the  next  section. 


Figure  4.1 


The  Enlistment  Process  and  Data  Sources 
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Validity  and  Reliability  of  the  Measures 

Each  measure  was  subjected  to  various 
reviews  and  tests  to  assess  its  validity  and  relia¬ 
bility  for  use  in  evaluating  the  field  experiment. 
More  findings  are  reported  in  considerable  detail 
as  each  measure  is  described  and  analyzed  in  the 
next  two  sections  of  this  chapter. 

Validity  in  its  broadest  terms  refers  to  the 
degree  to  which  each  chosen  measure  actually 
represents  that  characteristic  of  the  phenomenon 
under  study.  Does,  for  example,  reported  inten¬ 
tions  to  enlist  on  the  YATS  represent  a  valid 
measure  (i.e.,  leading  indicator)  of  an  eventual 
enlistment  decision?  Can  the  relationship  be 
"validated"  through  rigorous  statistical  procedures 
or  must  some  other  standards  be  used? 

Once  a  measure's  validity  has  been 
established,  then  its  reliability  or  extent  of  "con¬ 
sistency”  must  also  be  quantified.  Validity 
addresses  the  "truth"  of  a  measure  while  reliability 
addresses  the  measure's  consistency  in  per- 
fonnance  or  outcome  through  repeated  use. 
Returning  to  reported  intentions  to  enlist  for  a 
moment,  the  measure's  reliability  can  be  judged 
by  the  consistency  of  the  outcome  through 
replication  of  the  measure's  use  in  the  field  experi¬ 
ment  This  issue  is  addressed  in  more  detail  in 
the  final  section  of  this  chapter. 

WHY  OBSERVED  MEASURES  OF 
BEHAVIOR  WERE  CHOSEN 

Enlistment  contracts  were  chosen  because 
advertising  budget  strategies  can  be  partially  evalu¬ 
ated  by  their  relative  effect  on  military  enlist¬ 
ments.  The  number  of  contracts  was  chosen  as 
the  appropriate  measure  rather  than  enlistments 
because  of  the  latter's  highly  seasonal  component, 
a  phenomenon  influenced  largely  by  the  avail¬ 
ability  of  training  slots.  Furthermore,  any 


measure  using  enlistments  (represented  by  the 
date  of  entry  to  active  duty)  may  lag  for  as  long  as 
one  year  after  a  contract  is  signed.  This  lag  is  not 
only  a  function  of  training  seat  availability,  but 
also  of  the  high  school  graduation  cycle  and,  to  a 
lesser  extent,  of  youth  preferences  for  delayed 
entry  dates  to  active  duty.  To  avoid  these  largely 
administrative  factors  confounding  a  measure 
using  only  enlistments,  the  number  of  contracts 
was  chosen. 

Most  prior  research  on  recruiting  and  the 
enlistment  process  also  used  a  measure  of 
contracts  signed  as  the  dependent  variable.  These 
studies  have  entailed  both  econometric  analyses 
(e.g..  Army  Research  Institute,  RAND,  UCLA 
and  Duke  University)  as  well  as  experimental 
approaches  (e.g.,  WARC,  RAND)  and  studies 
employing  an  econometric  approach  analyzing 
historical  data  (e.g.,  Epps,  1971,  Hernandez, 
1979;  Goldberg,  1982;  Hanssens  and  Levien, 
1983;  Morey  and  McCann,  1980)  in  various 
controlled  experiments  (e.g.,  Carroll,  &  aL 
1985).  In  general,  these  studies  concluded  that 
marketing  efforts  are  statistically  related  to 
enlistment  contracts. 

On  the  face  of  it,  widespread  use  of  enlistment 
contracts  in  prior  research  lends  validity  to  its  use 
as  a  more  appropriate  measure  in  this  research 
project.  Contract  data  collected  during  the  period 
of  the  experiment  cannot,  however,  serve  as  a 
valid  predictor  of  the  long-term  consequences  of 
changed  advertising  budget  strategies  on  enlist¬ 
ments.  Other  measures  must  be  used  for  that 
purpose. 

Applicants  and  the  conversion  ratio  of 
applicants  into  contracts  represent  the  second 
and  third  observed  measures  of  behavior.  These 
measures  were  chosen  for  several  reasons.  First, 
the  number  of  applicants  who  take  the  qualifi¬ 
cation  test  is  less  constrained  by  enlistment 
standards  than  contracts.  Indeed,  not  all  those 


who  take  the  test  eventually  sign  contracts. 
Second,  test-taking  is  also  less  influenced  by 
direct  recruiter  efforts  than  are  contracts, 
suggesting  that  the  flow  of  applicants  may  be 
more  sensitive  to  the  effects  of  advertising. 
Third,  because  test-taking  is  one  of  the  earliest 
and  most  accurately  measured  indicators  of 
advertising's  effect  on  the  recruiting  process,  this 
variable  can  serve  as  a  leading  indicator  of 
advertising's  longer-term  effect  on  contracts. 

These  reasons  help  substantiate  the  choice  of 
applicants  and  the  related  applicant  conversion 
ratio  as  appropriate  and  valid  measures  for 
evaluating  the  effectiveness  of  the  various 
advertising  budget  strategies. 

Validating  the  Use  of  Applicant  and 
Conversion  Ratio  Measures 

As  is  evident  from  the  prior  discussion,  a 
considerable  body  of  research  documents 
enlistment  contracts  as  a  valid  measure  of 
advertising  effectiveness.  Yet,  a  notable  absence 
of  prior  research  attempting  to  correlate  applicant 
flows  and  advertising  spending  requires  a 
different  approach  to  validate  these  measures  as 
appropriate  for  the  field  experiment. 

As  part  of  the  validation  effort,  the  statistical 
relationship  between  enlistment  contracts,  appli¬ 
cants  taking  the  test,  and  advertising  spending 
levels  by  Service  was  investigated. 

DMDC  provided  monthly  data  on  applicants 
taking  the  test  and  enlistment  contracts,  for  each 
Service,  by  county  and  by  education  and  aptitude. 
These  data  sets  covered  the  period  October  1978 
through  March  1980.  Marketing  and 
environmental  data  (e.g.,  unemployment  rate, 
percent  black),  originally  developed  for  the 
Wharton-Navy  Field  Marketing  Experiment  for 
43  markets,  covering  the  period  October  1978 
through  September 


1979,  were  included.  These  data  were  combined 
with  additional  data  collected  on  the  same 
variables  for  the  remaining  period  October  1979 
through  March  1980. 

Since  the  statistical  models  estimated  involved 
the  use  of  lagged  variables  (specifically  contracts 
and  applicants),  only  data  for  the  first  quarter  of 
1980  were  used  to  examine  the  relationship 
between  advertising,  applicants  and  contracts. 
The  dependent  variables  consisted  of  male  appli¬ 
cants  taking  the  test  and  male  enlistment  contracts, 
both  expressed  on  a  per-capita  basis  (17-21-year- 
old  males).  These  dependent  variables  were 
further  disaggregated  by  education  and  by  a 
measure  of  general  ability. 

The  first  group  (dependent  variable)  disaggre¬ 
gated  were  either  seniors  or  High  School  Diploma 
Graduate  (HSDG)  males  who  performed  above 
average  on  national  norms  (see  Table  4.1)  for  die 
Armed  Forces  Qualification  Test  (AFQT).  The 
second  group  disaggregated  simply  consisted  of 
all  remaining  male  applicants  and  enlistment 
contracts. 

The  first  disaggregated  group  represented  the 
target  market  of  interest.  The  independent  or 
predictor  variables  included  dollar  expenditures 
for  Joint  and  Service-specific  advertising,  the 
number  of  recruiters  by  Service,  and  certain 
environmental  variables. 

The  results  of  the  analysis,  depicted  in  Table 
4.2,  show  that  both  Joint  and  Service-specific 
advertising  had  a  positive  effect  on  the  high- 
quality  group  of  applicants  and  contracts. 

Taken  in  total,  the  literature  and  results  of  the 
statistical  analysis  presented  here  provide 
considerable  evidence  that  the  chosen  measures  of 
observed  behavior  (i.e.,  contracts,  applicants  and 
the  conversion  ratio  of  applicants  to  contracts) 
possess  sufficient  validity  to  warrant  their  use  in 
the  evaluation  of  the  field  experiment. 


Table  4.1 


Percentage  Distribution  of  Civilian  Youth  Population 
by  AFQT  Category,  1980* 


AFQT 

Category 

Percentile  Score 

Range 

Percent  of  Civilian 

Youth  Population11 

I 

93-100 

8 

n 

65-92 

28 

mA 

50-64 

16 

nm 

31-49 

18 

IV 

10-30 

21c 

V 

1-9 

9d 

a  On  the  basis  of  AFQT  scores,  examinees  are  divided  into  six  categories  representing  a 
range  from  high  trainability  (Category  I)  to  low  trainability  (Category  V).  By  law,  test 
scores  below  10  (Category  V)  disqualify  an  individual  from  military  service.  Those 
scoring  between  the  10th  and  30th  percentiles  (Category  IV)  are  considered  by  the  Services 
to  require  a  longer  period  of  training  and  are  less  productive  in  jobs  requiring  a  high  level 
of  technical  skill.  There  is,  therefore,  a  20-percent  ceiling  on  the  enlistment  of  personnel  in 
Category  IV,  and  all  such  personnel  must  be  high  school  diploma  graduates. 


b  The  reference  population  is  based  on  the  testing  of  a  nationally  representative  sample  of 
young  people,  ages  18  to  23  (more  details  can  be  found  in  Profile  of  American  Youth: 
1980  Nationwide  Administration  of  the  Armed  Services  Vocational  Antitude  Battery. 
Department  of  Defense,  Office  of  the  Assistant  Secretary  of  Defense  (Manpower,  Reserve 
Affairs,  and  Logistics),  March  1982). 


c  Effective  1  October  1981,  the  number  of  Non-Prior  Service  (NPS)  enlistees  who  score  at 
or  above  the  tenth  percentile  and  below  the  thirty-first  percentile  on  the  AFQT,  may  not 
exceed  20  percent  of  the  total  number  of  NPS  enlistments  per  Fiscal  Year  (10  (JSC 
520(a)).  In  any  case,  a  person  who  is  not  a  high  school  graduate  may  not  be  accepted  for 
enlistment  with  an  AFQT  score  below  the  thirty-first  percentile  ( 10  (JSC  520(b)). 


d  Individuals  in  Category  V  do  not  meet  minimum  standards  for  enlistment. 


Table  4.2 

Correlations  between  Advertising  Efforts  and  Applicants/Contract  Measures 


APPLICANTS 


MALE  HSDG/AFQT  I-m  A: 

Total 

ARMY 

navy 

AIR  FORCE 

MARINES 

Service-Specific  Advertising 

+ 

+ 

0 

o 

o 

Other  Service  Advertising 

N/A 

+ 

0 

+ 

o 

Joint  Advertising 

+ 

+ 

o 

+ 

o 

ALL  OTHER  MALES: 

Total 

army 

NAYY 

AIRFORCE 

MARINES 

Service-Specific  Advertising 

o 

o 

o 

o 

• 

Other  Service  Advertising 

N/A 

- 

+ 

- 

o 

Joint  Advertising 

o 

o 

o 

o 

CONTRACTS 

MALE  HSDG/AFQT  I-ffl  A: 

Total 

army 

NAVY 

AIRFORCE 

MARINES 

Service-Specific  Advertising 

+ 

+ 

- 

+ 

Other  Service  Advertising 

N/A 

0 

- 

o 

o 

Joint  Advertising 

+ 

+ 

+ 

+ 

o 

ALL  OTHER  MALES: 

Total 

ARMY 

NAVY 

AIRFORCE 

MARINES 

Service-Specific  Advertising 

o 

. 

+ 

+ 

+ 

Other  Service  Advertising 

N/A 

o 

+ 

o 

+ 

Joint  Advertising 

o 

o 

+ 

+ 

+ 

+  =  significant  positive  -  *  significant  negative  o  =*  no  significant  N/A  =  Not  Applicable 
correlation  correlation  correlation 


Total — Sum  of  all  Service  advertising. 

Service-Specific  Advertising — Advertising  geared  only  to  promote  one  Service. 

Other  Service  Advertising  —  Advertising  promoting  a  Service  other  than  the  one  indicated  at 
the  top  of  the  column. 


Example:  •  There  is  no  significant  correlation  between  male  HSDG/AFQT  Mil  A  Air  Force 
applicants  and  Air  Force-specific  advertising. 

•  There  is  a  significant  positive  correlation  between  male  HSDG/AFQT  MIIA  Air 
Force  applicants  and  advertising  promoting  Services  other  than  the  Air  Force. 


WHY  REPORTED  BEHAVIOR  AND 
ATT1TUDINAL  MEASURES  WERE 
CHOSEN 

The  observed  behavior  measures  capture  only 
the  relatively  short-term  effects  of  advertising. 
Thus,  the  potentially  long  length  of  the  enlistment 
cycle  argues  for  additional  measures  assessing  the 
longer-term  effects  of  advertising.  For  example, 
prospects  may  talk  to  a  recruiter  and/or  have 
conversations  with  their  parents  about  military 
service  as  a  result  of  exposure  to  advertising. 
This  behavior  may  subsequently  contribute  to  an 
actual  enlistment  decision.  In  addition,  adver¬ 
tising  may  have  the  effect  of  changing  the 
prospect's  attitudes  toward  the  military,  that  may 
in  turn  result  in  a  favorable  enlistment  decision. 

Three  measures  were  selected  to  address  these 
issues:  reported  recruiter  contact,  reported  con¬ 
versations  with  parents,  and  stated  intentions  of 
joining  the  military. 

The  analysis  focused  on  firmly  establishing 
these  measures  as  valid  predictors  of  eventual 
enlistment  behavior.  Both  reported  behavior 
measures  (reported  recruiter  contact  and  reported 
conversations  with  parents)  and  attitudinal 
measures  (stated  intentions  of  joining  the  military) 
were  tested  for  their  ability  to  predict  actual 
enlistment  behavior  during  a  period  of  several 
years.  Moreover,  the  independent  and  potential 
interactive  effects  of  recruiter  contact,  conver¬ 
sations  with  parents  and  youths'  stated  intentions 
were  also  considered. 

All  three  measures  -  stated  intentions  to  enlist, 
reported  recruiter  contact,  and  conversations  with 
parents  by  potential  recruits  -  are  measured  by  the 
YATS,  conducted  regularly  since  1975.  Stated 
intentions  to  enlist  is  used  as  an  indicator  of  youth 
attitudes  toward  the  military  and  each  Service,  as 
is  reported  recruiter  contact.  Reported  recruiter 


contact  is  also  used  as  an  indicator  of  the 
effectiveness  of  the  recruiter  salesforce.  The 
specific  YATS  questions  are  described  as  follows: 

Conversations  with  Parents 

Conversations  with  parents  was  based  on  the 
responses  to  YATS  questions  in  1984  and  1983. 
Respondents  were  first  asked:  "Within  the  last 
year  or  so,  have  you  discussed  with  anyone  the 
possibility  of  serving  in  the  military?"  More  detail 
was  solicited  from  those  who  answered  "yes," 
including:  "With  whom  did  you  discuss  serving 
in  the  military?"  Respondents  could  indicate  that 
they  had  had  recent  discussions  about  joining  the 
military  with  either  or  both  parents.  Mentions  of 
"mother"  or  "father"  were  independently  recorded 
in  1984  and  1983. 

Conversations  with  Recruiters 

This  measure  included  share  conversations 
with  recruiters  from  the  Army,  Navy,  Air  Force 
and  Marine  Corps.  Conversations  with  recruiters 
was  based  on  the  YATS  question:  "Have  you 
ever  talked  with  any  military  recruiter  to  get 
information  about  the  military?"  Shares  of 
conversations  with  recruiters  were  derived  from 
those  who  responded  "yes"  to  the  question: 
"What  military  service  did  the  recruiter 
represent?"  Note  that  the  share  measure  may  sum 
to  more  than  100  percent  since  the  same  respon¬ 
dent  could  have  met  with  recruiters  from  more 
than  one  Service. 

Intentions  to  Enlist 

The  intention  measures  are  derived  from  a 
composite  of  two  YATS  survey  questions.  One 
question  asks  respondents  for  their  planned 


activities  in  the  next  few  years  (unaided  mention  - 
definite  and  probable  intentions).  The  second 
question  asks  respondents  to  indicate  their 
intentions  to  enlist  in  the  military  using  a  five- 
point  scale  ("definitely"  to  "definitely  not”).  The 
specific  questions  are  described  in  more  detail  as 
follows: 

Unaided  Mention  of  Joining  the  Military 
The  YATS  questionnaire  begins  with  general 
questions  about  the  youth's  demographic  and 
educational  background.  Soon  thereafter,  the 
youth  is  questioned  about  his  career  plans  without 
any  specific  mention  of  the  military.  A  response 
indicating  interest  in  a  military  career  indicates 
inclusion  of  the  military  in  the  respondent's  main 
choice  set  and  may  reflect  top-of-mind  awareness 
on  part  of  the  youth  about  military  career  alterna¬ 
tives.  The  measure  of  unaided  mention  of  joining 
the  military  is  based  on  the  YATS  question: 
"Now,  let's  talk  about  your  plans  for  the  next  few 
years.  What  do  you  think  you  might  be  doing?" 
The  response  records  whether  "Joining  the 
military/service"  was  mentioned  by  the  youth. 
This  measure  is  referred  to  as  "unaided  mention 
of  joining  the  military"  in  this  report 

Aided  Mention  of  Joining  the  Military 
Respondents  are  next  asked  about  their  responses 
to  specific  career  opportunities.  Hie  questions 
begin  with  the  mention  of  non-military  careers 
such  as  construction  worker  and  office  worker. 
Soon  thereafter,  a  mention  of  the  military  is  made 
in  the  question:  "How  likely  is  it  that  you  will  be 
serving  in  the  military?"  This  measure  is  referred 
to  as  "likelihood  of  joining  the  military." 

Composite  Likelihood  of  Joining  the 
Military 

The  respondents  are  probed  about  their  intention 
to  join  specific  Services.  A  measure  of  composite 


likelihood  of  joining  the  military  is  constructed 
based  on  the  response  to  these  questions.  The 
YATS  questions  are:  "How  likely  is  it  that  you 
will  be  serving  on  active  duty  in  the  Army/Air 
Force/Marine  Corps/Navy?"  If  the  answer  to  any 
of  these  four  questions  was  recorded  as 
"probably"  or  "definitely,"  the  respondent  was 
considered  to  have  a  favorable  attitude  toward 
serving  in  some  specific  branch  of  the  military. 
The  number  of  respondents  with  favorable 
attitudes  relative  to  the  total  number  of 
respondents  in  any  particular  cell  is  referred  to  as 
the  "composite  likelihood  of  joining  the  military." 
Shares  of  composite  likelihood  for  each  Service 
were  derived  by  taking  the  proportion  of  respon¬ 
dents  with  favorable  attitudes  toward  a  specific 
Service  to  the  total  respondents  having  favorable 
attitudes  toward  all  Services.  These  shares  do  not 
necessarily  add  to  100. 

WHAT  THE  ANALYSES  SHOW 
ABOUT  THE  REPORTED  BEHAVIOR 
MEASURES 

The  marketing  literature  supports  the  use  of 
these  measures  to  capture  the  longer-term  effects 
of  advertising.  Reported  behavior  measures  have 
been  used  in  several  studies  and  are  reviewed  in 
Silk  and  Kalwani  (1982).  Orvis,  at  RAND,  has 
conducted  extensive  analyses  using  stated 
enlistment  intentions  (1982;  1984).  His  results 
reveal  that  stated  enlistment  intentions  are  highly 
correlated  with  future  enlistment  behavior.  Orvis 
and  his  colleagues  also  report  that  recruiter  contact 
and  conversations  with  parents  discriminate 
between  individuals  with  positive  and  negative 
intentions.  Finally,  Bayus  (1985)  presents 
evidence  that  conversations  about  the  military  are 
statistically  related  to  changes  in  advertising 
expenditures. 


The  power  of  reported  behavior  and  atdtudinal 
measures  to  predict  enlistments  was  analyzed 
using  the  longitudinal  follow-up  to  the  Youth 
Attitude  Tracking  Study.  Matching  the  social 
security  number  of  respondents  for  each  survey 
wave  allowed  tracking  of  actual  enlistment 
behavior  and  comparison  to  the  reported  behavior 
and  atdtudinal  measures.  Orvis  and  his  col¬ 
leagues  at  RAND  assembled  these  data  for  the 
years  1976  -  1979,  encompassing  more  than 
23,000  respondents.  Test-taking  status  and  final 
enlistment  behavior  were  tracked  for  each 
individual  for  up  to  four  years,  a  sufficient  time 
interval  for  assessing  enlistment  activity  (Orvis, 
1982). 

Figures  4.2,  4.3,  and  4.4  summarize  the 
results  for  reported  recruiter  contact,  reported 
conversations  with  parents,  and  stated  enlistment 


intentions,  respectively.  The  cumulative  enlist¬ 
ment  rate  for  respondents  stating  they  had  face-to- 
face  recruiter  contact  is  about  five  percent  after  six 
months,  increasing  to  about  13  percent  after  42 
months  (Figure  4.2).  For  respondents  not  having 
recruiter  contact,  the  cumulative  enlistment  rate  is 
signficantly  lower  over  time  (seven  to  eight 
percent  after  42  months). 

Reported  conversations  with  parents  follows  a 
similar  pattern  (Figure  4.3).  This  measure  is  a 
better  predictor  over  time  than  the  recruiter  contact 
measure  (the  difference  between  the  reported 
conversations  with  parents  and  no  conversation 
with  parents  curves  (Figure  4.3)  is  greater  at  any 
point  in  time  than  the  difference  between  the 
reported  recruiter  contact  and  no  recruiter  contact 
curves  (Figure  4.2)). 


Figure  4.2 

Cumulative  Enlistment  Rate  by  YATS  Respondents 
Reported  Face-to-Face  Recruiter  Contact 
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Figure  4J 


Cumulative  Enlistment  Rate  by  YATS  Respondents 
Reported  Conversations  with  Parents 
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Source:  Longitudinal  Follow-Up  to  1 976-79  YATS 


Orvis  (1982)  discusses  these  measures  in 
greater  detail.  Additional  analyses  indicate  that 
the  single  measure  of  stated  intentions  to  enlist 
also  performs  very  well.  The  enlistment  rate  per 
six-month  period  for  respondents  who  have  not 
yet  enlisted  and  who  have  positive  intentions  of 
joining  the  military,  ranges  from  about  29  percent 
after  six  months  to  about  1.5  percent  after  42 
months.  The  enlistment  rate  for  respondents  with 
negative  intentions  is  significantly  lower  (Figure 
4.4). 

Table  4.3  presents  the  marginal  effects  of 
these  variables  on  final  enlistment  behavior  and 
on  test-taking  activity  (without  eventual 
enlistment).  These  values  give  an  estimated 
elasticity  of  the  reported  measures.  For  example, 
a  one-percent  increase  in  the  target  population 
having  conversations  with  parents  would  result  in 
a  0.66  percent  increase  in  enlistments.  These 
results  further  indicate  that  of  the  three  variables, 


stated  intentions  is  the  most  significant  predictor 
of  eventual  enlistment  and  that  conversations  with 
parents  is  a  better  predictor  than  recruiter  contact. 
Details  are  described  in  Bayus  and  Carroll  (1985). 

As  a  further  test  of  the  predictive  power  of  the 
three  selected  measures,  the  possibility  of 
interactive  effects  between  them  was  investigated. 
Table  4.4  shows  the  enlistment  rates  of  YATS 
respondents  who  indicated  positive  and  negative 
intentions  to  join  the  military.  These  were  further 
analyzed  with  respect  to  their  behavior  concerning 
conversations  with  parents  regarding  the  military 
and  recruiter  contacts. 

Depending  upon  how  they  report  their 
intentions  toward  possible  enlistment  and  on  any 
preliminary  contacts  with  recruiters  and 
discussions  with  parents,  the  data  on  Table  4.4 
reveal  a  marked  difference  in  subsequent 
enlistment  rates. 


m 


Figure  4.4 

Enlistments  Per  Hundred  YATS  Respondents  Aged  17.5 


Source  Longitudinal  Follow-Up  to  1976-79YATS 

Moreov  or,  visual  inspection  of  the  data  reveal  behaviors  and  dispositions  toward  the  military  as 
a  clear  pattern  between  the  YATS  respondent's  a  leading  indicator  of  subsequent  enlistment, 
stated  involvement  in  the  enlistment  process  and  Although  not  immediately  apparent, 

subsequent  enlistment.  Specifically,  the  more  differences  in  the  enlistment  rates  across  the 

active  the  individual's  search  process  and  the  YATS  response  groups  suggest  that  search 

more  positive  his  orientation,  the  more  likely  behavior  and  attitudes  reinforce  each  other.  In  the 

eventual  enlistment  becomes.  These  data  appear  right  combination,  these  measures  can  lead  to  an 

to  confirm  one’s  intuitive  notions  about  prior  even  greater  enlistment  rate.  For  example,  the 


Table  4.3 


Estimated  Elasticities  of  Reported  Behavior 
and  Attitudinal  Measures 


Reported  Measures 
of  Behavior: 


Attitudinal  Measure: 
Intentions  to  Join  the  Military 


Enlistment  Rates  by  Prior  Intention 
to  Join  the  Military 


19.2% 


5.4% 


Total  enlistment  rate  regardless  of  intention: 


Source;  Longitudinal  Follow-up  to  1976-1979  YATS 


9.2% 


subsequent  enlistment  rate  on  Table  4,4  for  the 
YATS  respondent  group  who  spoke  with  their 
parents,  contacted  a  recruiter  and  had  a  positive 
intention  to  enlist,  amounts  to  26.6  percent.  This 


is  more  than  twice  the  rate  (10.4  percent)  for 
those  in  the  same  parent  conversation  and 
recruiter  contact  category,  but  expressing  negative 
intentions  toward  enlisting. 


SENSITIVITY  OF  THE  MEASURES 


Measures  of  Statistical  Error 


Evaluating  the  DoD  Advertising  Mix  Test 
requires  assessing  whether  differences  in 
advertising  policy  (budget  levels  and  mixes)  result 
in  differences  in  enlistment  behavior.  As 
reviewed  in  the  prior  section,  the  measures  have 
been  substantiated  for  their  validity.  Now  the 
level  by  which  the  measures  must  differ  before 
the  resulting  differences  can  be  attributed  to  adver¬ 
tising  policy  (within  appropriate  error  tolerances) 
must  be  established.  This  range  of  differences  is 
referred  to  as  a  measure  of  sensitivity. 

Reliability  of  Measures 

The  reliability  of  a  measure  refers  to  the 
sensitivity  of  the  measure  in  capturing  the  effects 
on  enlistment  of  different  advertising  policies. 
Knowing  how  much  the  measures  must  change, 
to  yield  a  statistically  significant  result,  enables  an 
assessment  of  the  relative  power  of  the  measures. 
The  reliability  of  the  measures  can  then  be 
inferred  by  assessing  the  likelihood  of  actually 
observing  this  range  of  movement  A  measure 
which  must  double  or  triple  to  yield  statistically 
significant  results  is  less  sensitive,  and  thus  less 
reliable,  than  one  which  must  change  only  10  or 
20  percent. 

The  results  of  several  studies  using  monthly 
and  quarterly  historical  time  series  data  (e.g., 
Fernandez,  1979;  Goldberg,  1982;  Hanssens  and 
Leiven,  1983;  Morey  and  McCann,  1980)  and  an 
experimentally  induced  variation  (Carroll,  si  si. 
1985)  have  revealed  that  the  observed  behavior 
measures  are  very  sensitive  to  changes  in 
advertising  and  recruiter  efforts. 


Before  discussing  the  results  of  the  sensitivity 
analyses,  the  statistical  concept  of  enor  requires 
elaboration. 

No  experiment,  no  matter  how  carefully 
controlled,  can  reduce  the  chances  of  error  to  zero 
within  reasonable  cost  boundaries.  Conducting  a 
DoD  advertising  experiment  with  the  chances  of 
error  reduced  strictly  to  zero  would  require 
creating  four  separate  United  States  of  America, 
conducting  the  experiment  over  an  extensive  time 
period,  and  gathering  complete  data  from  every 
member  of  the  youth  population  under  study. 
This  is  obviously  impossible.  Instead, 
researchers  in  both  the  physical  and  social 
sciences  rely  on  concepts  of  statistical  inference  to 
set  appropriate  error  tolerances  for  interpreting 
experimental  data. 

Two  types  of  error  must  be  avoided  when 
designing  and  evaluating  a  comparative  study 
(Fleiss,  1981).  The  first  error,  called  Type  I 
error,  occurs  when  the  differences  in  responses 
under  examination  are  declared  to  be  real,  when  in 
fact  there  are  no  differences.  For  example,  this 
type  of  error  could  arise  if  a  difference  in 
enlistment  behavior  were  attributed  to  an 
advertising  policy  when  no  such  difference 
existed.  More  specifically,  this  error  would  occur 
if  an  observed  decline  in  enlistments  in  a  treatment 
cell  were  attributed  to  the  advertising  policy  in  that 
cell,  when  in  fact  the  decline  in  the  treatment  cell 
was  no  different  from  that  in  other  treatment  cells. 
This  kind  of  error  is  generally  prevented  by 
simply  setting  the  statistical  test  at  a  small  level  of 
probability  (alpha  level)  such  as  0.10,  0.05,  or 
0.01 .  This  kind  of  control  is  not  entirely  adequate 


since  populations  underlying  the  observed 
responses  will  generally  differ  to  some  extent. 
These  differences  can  be  substantially  reduced  by 
careful  population  or  market  matching  proce¬ 
dures.  This  has  been  done  in  the  cell  design  of 
this  test 

The  second  kind  of  statistical  error,  Type  n, 
occurs  when  the  two  responses  under 
examination  are  not  declared  significantly 
different  when  in  fact  they  are  different.  This 
type  of  error  would  arise  in  the  test  if  the  same 
effects  of  advertising  on  enlistments  were 
attributed  to  treatment  cells  when,  in  fact,  the 
advertising  policies  in  these  cells  led  to  different 
enlistment  behavior.  Such  an  error  is  less  serious 
when  the  responses  differ  by  a  small  amount  It 
becomes  critical  when  the  difference  is  large.  The 
researcher  can  control  Type  II  error  by  specifying 
what  difference  is  of  sufficient  importance  to  be 
detected,  and  what  the  desired  probability  is  of 
detecting  it  This  probability,  denoted  as  one  min¬ 
us  beta,  is  called  the  power  of  the  statistical  test 
Beta  is  the  probability  of  failing  to  declare  the 
specified  difference  to  be  statistically  important 

The  sensitivity  of  the  reported  behavior  and 
attitudinal  measures  must  be  evaluated  for  pre¬ 
determined  alpha  and  beta  values.  Cohen  (1977) 
suggests  that  in  the  typical  case.  Type  I  error  is 
approximately  four  times  as  serious  as  Type  II 
error.  Based  on  the  experimental  design  of  the 
DoD  Advertising  Mix  Test  the  costs  for  Type  I 
and  Type  II  errors  can  be  roughly  computed. 

Type  I  error  (declaring  that  a  difference 
between  treatment  conditions  exists  when  there  is 
no  real  difference)  would  result  in  a  maximum 
annual  cost  of  $53  million  to  the  Department  of 
Defense.  This  figure  is  derived  from  the 
difference  between  the  test  cell  with  the  highest 
cost  (Cell  White  with  a  total  advertising  budget  of 
$84  million)  and  the  cell  with  the  lowest  cost  (Cell 


Blue  with  a  total  advertising  budget  of  $31 
million).  Determining  the  maximum  cost  of  a 
Type  II  error  (declaring  that  no  difference 
between  treatment  conditions  exists  when  there  is 
a  significant  difference)  requires  developing  the 
cost  of  recapturing  lost  enlistment  contracts.  That 
loss  would  be  the  result  of  choosing  a  cell  with 
less  advertising  than  would  be  efficient  and 
effective. 

These  results  imply  that  the  cost  of  Type  I  and 
Type  II  errors  are  about  the  same  for  the  DoD 
Advertising  Mix  Test.  Both  types  of  errors  need 
to  be  prevented  at  similar  levels  of  statistical 
assurance.  This  means  a  tighter  constraint  for 
Type  II  errors.  The  sensitivity  analyses  used 
various  values  for  alpha  and  beta  to  obtain  a 
sensitivity  range  for  each  selected  measure. 

Sensitivity  of  Reported  Behavior  and 
Attitudinal  Measures 

The  sensitivity  of  reported  recruiter  contact, 
reported  conversations  with  parents,  stated 
enlistment  intentions,  and  the  interactive  effect 
between  recruiter  contact  and  parent  conversations 
was  computed  by  using  the  pre-intervention 
survey  (1983  Youth  Attitude  Tracking  Study) 
responses  as  a  base.  This  information  was 
compared  to  the  results  from  the  post-intervention 
survey  (YATS  study  done  after  the  experiment)  to 
determine  the  statistical  significance  of  the 
changes  in  the  measures.  The  sensitivity  of  each 
measure  differs  by  treatment  cell  and  by  sample 
size.  With  the  relative  changes  in  baseline 
responses  known,  the  likelihood  of  those 
differences  can  be  assessed. 

Prior  to  evaluating  the  sensitivity  of  the  YATS 
measures  (in  responding  to  changes  in  recruitment 
advertising),  several  issues  had  to  be  considered. 
The  first  concerns  the  effects  of  geographic  and 


Figure  4.5 

General  Approach  for  Calculating  the  Sensitivity  of 
Evaluation  Measures 
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sampling  differences  between  the  cells.  For 
example,  one  test  cell  may  have  a  greater 
proportion  of  its  population  exposed  to  recruiters 
than  another  test  cell.  In  another  case,  because  of 
differences  in  samples,  the  level  of  recruiter 
contact  may  be  25  percent  of  respondents  in  one 
sample  while  26  percent  in  another  sample.  The 
second  issue  to  be  considered,  regarding 
evaluating  the  sensitivity  of  the  YATS  measures, 
concerns  time  trends.  Different  values  for  a 
measure  may  be  obtained  within  a  market  at 
various  points  in  time.  For  example,  recruiter 
contact  in  a  cell  may  change  from  the  1983  to  the 
1984  YATS  results.  Consequently,  differences  in 
the  YATS  measures  were  investigated  to  ensure 
that  any  observed  differences  were  actually  attri¬ 
butable  to  advertising  policy  differences.  The 
steps  to  accomplish  this  goal  are  represented  in 
Figure  4.5. 


Table  4.5  contains  the  pre-intervention  (prior 
to  experiment)  responses  for  the  1983  YATS  by 
test  cell  for  each  of  the  evaluation  measures. 
These  results  were  positive  in  that,  with  the 
exception  of  enlistment  intentions  which  is  mildly 
significant  for  Cell  Blue,  none  of  the  main  effect 
measures  were  significantly  different  across  the 
treatment  cells.  The  interactive  effect  between 
conversations  with  parents  and  recruiter  contact 
was  significantly  higher  in  Cell  Blue  (and 
correspondingly  lower  in  Cell  Yellow)  indicating 
that  the  1984  YATS  responses  required 
adjustment  when  studying  this  interactive  effect. 

Sampling  differences  for  demographic  and  co¬ 
variate  variables  were  evaluated  by  cross 
tabulating  responses  across  treatment  cells  for  the 
1983  YATS.  With  the  exception  of  race,  these 
results  revealed  no  major  differences.  Results  for 
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Table  4.5 

Pre- Intervention  Statistics  for  Evaluation  Measures  by  Cell 


VARIABLE 

CELL  A1 
YELLOW 

CELL  B 
BLUE 

CELL  C 
GREEN 

CELLD 

RED 

POOLED 

ACROSS 

CELLS 

Positive  Intentions 

27.1% 

*23.1% 

28.5% 

29.1% 

26.0% 

Conversations  with 
Parents 

26.5% 

27.1% 

31.5% 

25.7% 

27.4% 

Recruiter  Contact 

43.5% 

44.4% 

46.0% 

43.0% 

44.1% 

Interaction  between 
Conversations  with 
Parents  and  Recruiter 
Contact  Effects 

**16.3% 

**18.7% 

19.9% 

16.6% 

17.5% 

Sample  Size 

640 

690 

387 

415 

Source:  1983  YATS  Weighted  Responses  *  Significant  difference  from  pooled  value  U  0.10  level 

•*  Significant  difference  from  pooled  value  at  0.05  level 

Reading  the  Table:  In  the  pre-intervention  YATS  survey,  27.1%  of  Cell  Yellow’s  respondents 
had  positive  intentions.  This  compares  to  Cell  Blue's  23.1%  positive 
intentions. 


the  race  variable  indicate  a  higher  proportion  of 
Caucasian  respondents  in  Cell  Green.  Thus,  the 
1984  YATS  population  statistics  for  this  variable 
required  adjustment  for  the  demographic  variable. 

To  establish  the  existence  of  a  time  trend, 
detailed  calculations  of  the  response  ranges  were 
made.  These  computations  identify  the  1984 
YATS  response  level  necessary  in  the  control  cell 
(Cell  Yellow)  to  infer  statistically  the  existence  of 
an  overall  time  trend  for  each  measure  (see  Table 
4.6).  Positive  intentions  to  enlist  in  the  military 
must  change  by  19  percent  from  its  1983  pre- 
intervention  (baseline)  response  (26.0  percent); 
conversations  with  parents  must  change  by  18 
percent  from  its  1983  baseline  response  (27.4 
percent);  and  recruiter  contact  must  change  by  13 
percent  from  its  1983  baseline  response  (44.1 
percent)  for  Type  I  and  Type  II  error  of  0.05. 


Table  4.6  presents  the  percent  change  required 
to  conclude  that  statistically  significant  treatment 
differences  exist  for  each  of  four  different 
variables.  These  changes  concern  the  pre¬ 
intervention  1983  YATS  responses  and  their  1984 
post-intervention  levels.  Various  Type  I  and 
Type  II  error  probabilities  are  included.  For 
example,  when  alpha  (Type  I  error  tolerance) 
equals  0.05  and  the  tolerance  for  Type  I  error  is 
the  same  as  for  Type  II  error,  stated  enlistment 
intentions  must  either  increase  from  the  base  of 
0.26  (  i.e.,  26.0  percent  of  respondents  had 
positive  intentions  of  joining  the  military)  to  0.32 
or  decrease  to  0.20  -  a  change  of  6.0  percentage 
points.  The  6.0  percentage-point  increase  or 
decrease  required  is  about  23  percent  of  the 
original  26.0  percent  response  level  observed  in 
1983. 


Table  4.6 


Percent  Change  in  the  1983  Pre- Intervention 
Measure  Required  for  Statistical  Significance 
between  Advertising  Treatments 


Type  I 

0.1 

0.05 

0.01 

Pre- 

Intervention 

Variable  Type  11 

0.1 

0.2 

0.4 

0.05 

0.10 

02 

0.01 

0.02 

0.05 

Responses 

Positive  Intentions 

19 

17 

13 

23 

21 

18 

28 

27 

25 

26.0% 

Conversations  with 

Patents 

18 

15 

12 

21 

19 

17 

27 

25 

24 

27.4% 

Recruiter  Contact 

13 

11 

9 

15 

14 

12 

19 

18 

17 

44.1% 

Interactions  between 
Conversation  with 
Parents  and  Recruiter 
Contact  Effects 

22 

20 

16 

26 

24 

21 

33 

32 

30 

17.5% 

SUMMARY  AND  CONCLUSIONS 

The  set  of  evaluation  measures  selected  to 
assess  the  short-  and  long-term  contributions  of 
various  advertising  mixes  toward  the  attainment 
of  the  Services'  enlistment  requirements  are  as 
follows: 

Observed  Behavior  Measures 

•  Quantity  and  quality  of  applicants  by 
Service 

•  Quantity  and  quality  of  contracts  by 
Service 

•  Conversion  ratios  of  applicants  into 
contracts 


Reported  Behavior  Measures 

•  Reported  recruiter  contact 

•  Reported  conversations  with  parents 

Attitudinal  measures 

•  Reported  intentions  to  enlist  in  the  military 

•  Reported  strength  of  intentions 

Each  of  these  measures  has  been  substantiated 
by  existing  literature  and  additional  testing.  All 
appear  to  be  valid  measures  for  purposes  of 
evaluating  the  experiment  according  to  the  criteria 
established.  These  measures  have  also  been 
found  sufficiently  sensitive  to  varied  advertising 
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mixes.  This  allows  for  policy  recommendations 
based  on  statistical  differences  between  the 
various  treatment  conditions. 

To  evaluate  correctly  the  data  provided  by 
these  measures  the  following  adjustments  and 
actions  were  required: 

•  Adjustments  for  the  variations  in  sample 
composition  within  certain  cells  were 
implemented.  In  particular,  Cell  Green 
required  adjustment  for  demographic 
differences  (i.eM  race). 


Adjustments  to  certain  cells  for  some 
evaluation  measures  were  made  to  allow 
for  correct  inferences.  For  example,  to 
examine  the  effects  of  advertising  using 
the  interaction  term  between  recruiter 
contact  and  parent  conversations.  Cells 
Yellow  and  Blue  were  adjusted. 

Type  II  error  constraints  must  be 
stringently  considered  in  evaluating  test 
results. 
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CHAPTER  5 


ANALYTICAL  APPROACH 


RESEARCH  PERSPECTIVE 

This  chapter  reviews  our  approach  to 
analyzing  the  data  from  the  DoD  Advertising  Mix 
Test  As  these  data  arrived,  they  were  checked, 
coded,  coHated  and  aggregated.  Appendix  D 
provides  a  description  of  the  data  collected  during 
the  test  Once  assembled,  validated  and  aggre¬ 
gated  into  a  data  base,  these  data  constituted  a 
comprehensive  quantitative  representation  of  the 
performance  of  the  recruiting  system  during  the 
period  of  the  test  (FY  1984). 

Earlier  chapters  of  this  report  have  discussed 
the  motivation  and  need  for  the  test,  its  objective, 
criterion  for  evaluation  and  specific  measures  of 
recruiting  system  performance  -  as  well  as  its 
actual  field  implementation.  This  chapter 
discusses  the  analytic  approach  and  quantitative 
techniques  used  to  evaluate  the  experiment. 
Specifically,  this  chapter  reviews  the  data 
elements,  the  level  of  aggregation,  and  the 
mathematical  representations  or  formulations 
used.  The  next  two  sections  of  this  chapter 
discuss  these  issues  for  observed  behavior 
measures  (enlistment  contracts  and  applicants), 
and  reported  behavior  and  attitude  measures. 

Two  general  observations  are  noted  before 
discussing  these  specific  measures,  however. 
First,  the  overall  assessment  of  the  experiment 
requires  investigation  across  measures.  Second, 
all  measures  of  recruiting  performance  declined 
during  the  period  of  the  test. 

The  need  to  look  across  measures  arises 
because  the  military  manpower  recruiting  system 


is  large  and  complex.  A  large  number  of  factors 
affect  it  and,  hence,  affect  any  quantitative 
representation  of  it.  Managerially  useful 
advertising  policy  guidelines  can  be  developed 
given  these  system  complexities  by  (1)  using 
multiple  measures  of  recruiting  system 
performance;  (2)  placing  more  emphasis  on 
consistency  among  a  variety  of  analyses;  and 
(3)  focusing  on  determining  the  direction  and 
ranges  of  advertising  expenditures,  rather  than  on 
developing  precise  and  static  decision  rules  for 
advertising  spending. 

Second,  the  measures  selected  for  evaluating 
the  enlistment  process,  both  applicants  and 
contracts,  substantially  and  consistently  declined 
from  FY  1983  to  FY  1984.  Every  measure  of  the 
enlistment  process  observed  during  the  period  of 
the  experiment  decreased  substantially  in  the 
matched  subset  of  the  control  cell  (Cell  Yellow) 
and  in  the  control  cell  as  a  whole.  Hence,  where 
no  changes  in  advertising  were  made,  the 
recruiting  system  was  still  less  productive  in  the 
experimental  year  than  in  the  prior  year.  For 
example,  aggregate  unemployment  levels 
decreased  during  this  period  and  entry  level 
military  pay  did  not  keep  pace  with  the  increase  in 
civilian  income  levels.  Both  of  these  factors  made 
recruiting  more  difficult. 

Table  5.1  provides  the  percent  change  in  the 
evaluation  measures  observed  in  the  subset  of  the 
control  cell  (Cell  Yellow)  between  FY  1983  and 
FY  1984.  It  also  indicates  the  change  in  the 
unemployment  rate.  The  overall  decline  in  these 
measures  during  the  period  of  the  test  has  several 
implications. 
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Table  5.1 


Percent  Change  in  Measures  of  the  Enlistment  Process 
Control  Cell  1983  to  1984 


Measure 

Percent  Change 

Total  Enlistment  Contracts 
(DoD  wide) 

-4.4 

High  School  Graduate  &  Senior 

Enlistment  Contracts  (DoD) 

-4.5 

AFQT  I-1HA  Enlistment 

Contracts  HSDG  (DoD)* 

-9.2 

Total  First  Applicants  (DoD  wide)** 

-19.8 

High  School  Graduate  &  Senior 

First  Applicants  (DoD) 

-20.6 

AFQT  I-IHA  First  Applicants 

HSDG  (DoD) 

-24.5 

Reported  Recruiter  Contact  (DoD) 

-16.1 

Reported  Conversations  with  Parents 
about  Enlisting  in  Military 

-18.8 

Unaided  Mention  of  Joining  Military 

-21.7 

Aided  Likelihood  of  Joining  Military 

- 15.1 

Aided  Composite  Likelihood  of  Serving 
in  One  or  More  Military  Services 

-8.3 

Overall  Unemployment  Rate 

- 19.6 

*  AFQT  I-IIIA:  individuals  scoring  at  or  above  the  50th  percentile  on  the  Armed 
Forces  Qualification  Test. 

**  Individuals  taking  the  non-institutional  Armed  Services  Vocational  Aptitude 
Battery  test  sequence  for  the  first  time. 


Assessments  of  the  effects  of  the  level  and  mix  of 
advertising  on  the  enlistment  process  were  made 
by  comparing  the  performance  of  these  measures 
in  each  cell  with  its  performance  in  the  control 
cell.  In  addition,  the  pervasive  decline  in  these 
measures  clearly  demonstrates  a  sharp  movement 
in  the  overall  system. 

The  DoD  Advertising  Mix  Test  was  designed 
to  evaluate  recruiting  advertising  policy  and  was 
not  designed  to  explain  fully  all  changes  in  the 
recruiting  system.  The  sharp  change  in  system 
performance  during  the  year  of  the  test  required 
more  focus  on  identifying  the  advertising-related 
changes  in  each  measure  of  performance  and 
allowed  less  focus  on  systematically  and  statisti¬ 
cally  relating  changes  in  one  measure  with 
changes  in  others.  As  a  result,  the  measures  were 
investigated  in  parallel  rather  than  in  sequence. 
That  is,  independent  analyses  were  performed  on 
the  various  measures  at  the  same  time  rather  than 
developing  a  sequential  analyses  plan  to  test 
results  across  measures  in  an  a  priori  sequence. 

OBSERVED  BEHAVIOR 

Because  of  the  design  of  the  experiment, 
cross-sectional  regression  models  were  used  to 
test  the  following  hypothesis:  Various  advertising 
treatments  produce  differing  responses  which  can 
be  measured  by  the  number  of  contracts  and 
applicants  (observed  behavior).  Each  ADI  in  the 
experiment  was  used  as  a  single  observation  for 
these  cross-sectional  regressions.  All  observa¬ 
tions  in  these  regressions  were  annual.  The 
various  observed  behaviors  were  modeled  as  a 
function  of  the  advertising  treatment  and  other 
explanatory  factors.  Dummy  variables  were  used 
to  indicate  the  advertising  treatments  instead  of 
actual  advertising  expenditures  on  Joint  and 
Service-specific  advertising  because  categorical 


variables  are  better  suited  to  the  experiment's  four 
cell  design. 

The  cross-sectional  models  using  all  the  ADIs 
were  supplemented  by  models  based  on  ADIs 
from  pairs  of  test  cells.  These  models  test  the 
hypothesis  that  the  cells  in  the  given  pair  exhibit 
different  responses  because  of  their  respective 
advertising  treatments.  Of  particular  importance 
is  the  comparison  of  each  test  cell  with  the  control 
cell.  These  "pair-wise"  analyses  were  performed 
only  with  1984  data  and  the  changes  from  1983  to 
1984. 

Formulation  Issues 

The  models  estimated  for  the  DoD  enlistment 
contracts  can  be  compared  with  several  other 
recent  models  that  estimate  enlistment  supply  for 
various  Services.  These  other  models  generally 
have  used  contracts  or  accessions  as  dependent 
variables.  The  following  comparisons  highlight 
five  key  formulation  issues. 

Goals 

Several  recent  models  of  enlistment  supply, 
including  work  by  WARC  with  the  Navy 
Recruiting  Command,  have  included  goals. 
Goals  were  not  explicitly  used  in  this  test's 
models  for  several  reasons.  First,  data  across 
Services  were  not  consistent.  Indeed,  the  Air 
Force  apparently  only  established  accession  goals 
during  the  period  of  the  test  whereas  other 
Services  also  established  contract  goals.  Second, 
the  effect  of  goals  on  contracts  is  considerably 
more  difficult  to  evaluate  when  several  Services 
are  involved.  For  example,  are  the  effects  of  the 
Army's  goals  confined  to  the  Army's  per¬ 
formance  or  do  the  Army’s  goals  impact  all 
Services?  Similarly,  how  should  changes  in 
recruitment  standards  be  incorporated  into  goal 


formulations?  Thus,  the  effects  of  goals  and 
standards  quickly  lead  to  the  identification  of 
alternative  formulations.  However,  the  complex¬ 
ity  of  the  recruiting  system  does  not  permit  dis¬ 
crimination  among  these  different  formulations. 

Another  reason  for  not  using  goals  in  the 
models  concerns  the  broad  and  general  decline  of 
the  recruiting  system's  performance  during  the 
test  period  --  the  overall  Delayed  Entry  Program 
(DEP)  pool  declined  significantly  during  FY 
1984.  This  factor  most  likely  reduced  the  impact 
of  goals  on  measures  such  as  contracts.  While 
goals  would  clearly  bind  against  accessions  in 
some  cases,  their  effect  on  contracts  and 
applicants  is  expected  to  have  been  reduced. 

Another  factor  in  eliminating  the  use  of  goals 
concerns  the  argument  that  they  would  be 
expected  to  have  a  progressively  lower  impact. 
Whereas  goals  might  be  argued  to  have  an  effect 
on  contracts,  their  effect  on  intentions  to  join  the 
military  is  dubious  at  best  Finally,  alternate 
formulations  concerning  the  impact  of  goals  were 
tested  in  several  models.  These  efforts  did  not 
materially  alter  the  nature  of  the  results. 


Lags 


Distributed  lag  formulations  for  variables  such 
as  unemployment  and  advertising  have  been 
widely  used  in  military  enlistment  supply  models. 
These  models,  which  use  monthly  data,  greatly 
increase  the  number  of  observations  and  the 
associated  degrees  of  freedom.  Unfortunately, 
the  increase  in  observations  is  countered  by  their 
serial  correlation.  Because  of  leads  and  lags  in 
the  recruiting  process,  the  number  of  contracts 
signed  in  a  given  month  is  frequently  serially 
correlated  with  the  number  of  signed  contracts  in 
previous  months.  In  addition,  monthly  models 
must  account  for  independent  variables  and 
seasonality.  This  frequently  renders  the  interpre¬ 
tation  of  coefficients  and  findings  difficult. 


The  models  employed  in  this  study  are 
generally  cross-sectional  models  using  annual 
data.  Distributed  lag  formulations  are  not  appro¬ 
priate  for  such  models  since  a  time  sequence  is 
not  considered.  However,  most,  if  not  all,  lag 
effects  are  picked  up  within  a  few  months  in  the 
distributed  lag  models  reported  in  military 
recruiting  studies.  Consequently,  the  impact  of 
various  independent  variables,  including  advertis¬ 
ing,  is  reflected  in  the  annual  data.  In  addition, 
different  rate  changes  in  unemployment  have  been 
directly  addressed  in  the  annual  change  models. 


Interaction  Terms 

Numerous  interactions  among  variables  are 
possible  and  likely  in  the  recruiting  system.  For 
example,  recruiting  effort,  unemployment  and 
advertising  may  be  highly  interactive  in  the 
recruiting  system.  Not  enough  is  known  about 
the  recruiting  system  to  identify,  formulate, 
specify  and  reliably  estimate  each  possible 
interaction  as  a  separate  term  in  a  model. 
Therefore,  a  model  formulation  that  allows  for 
interaction  among  the  variables  has  been 
employed.  The  response  variable  is  modeled  as 
the  product  of  functions  of  the  independent 
variables.  Such  formulations  have  been  exten¬ 
sively  employed  in  marketing  modeling  work. 


Demand  Constraints 

It  is  generally  believed  that  the  Services 
administratively  control  the  input  of  less  qualified 
applicants  but  accept  all  the  available  but  limited 
supply  of  those  better  qualified  applicants.  For 
reasons  of  trainability,  performance  and  retention, 
the  Services  prefer  those  applicants  who  are  high 
school  graduates  (or  seniors  who  will  graduate 
prior  to  entry  on  active  duty)  and  who  also  score 
in  the  top  half  of  the  aptitude  range  on  the  Armed 
Forces  Qualification  Test  (AFQT). 
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Many  models  of  enlistment  supply  have  been 
formulated  under  the  assumption  that  supply- 
generating  initiatives,  such  as  recruiting  and 
advertising  resources  and  bonuses,  affect  only  the 
input  of  the  supply-limited  group  of  better 
qualified  applicants.  In  practice,  however,  the 
demand  for  the  less  qualified  applicants  is  also 
affected  by  changes  in  recruiting  resources  and  by 
the  Services'  efforts  to  meet  their  active  duty 
manning  requirements. 

The  dynamics  of  the  marketplace,  coupled 
with  the  Services’  own  manpower  policies  and 
practices,  can  produce  what  appear  to  be  counter¬ 
intuitive  results.  That  is,  the  input  of  less 
qualified  applicants  can  increase  along  with  those 
better  qualified  as  more  resources  are  injected  into 
the  recruiting  system.  Short-  and  intermediate- 
term  factors  such  as  training  seat  availability  and 
an  end-strength  shortfall,  for  instance,  may 
encourage  the  Services  to  relax  temporarily  their 
administrative  controls  and  allow  a  greater  input 
of  the  less  qualified  along  with  those  better 
qualified  applicants. 

If  only  the  preferred  group  of  better  educated 
and  above  average  aptitude  enlistment  applicants 
and  contracts  were  included  in  the  analysis,  in  our 
judgment  the  field  experiment  would  have  been 
incomplete,  and  possibly  subject  to  invalid 
results.  The  Services  cannot  in  reality  achieve  a 
global  optimum  that  ensures  at  all  times  a  perfect 
balance  between  resource  allocation  initiatives, 
force  manning  objectives  and  accession  policies. 
Thus,  it  would  be  unwise  to  assume  in  this  experi¬ 
ment  that  applicant  and  contract  flows  of  the  less 
preferred  (and  at  least  partially  demand  con¬ 
strained)  should  be  ignored  as  irrelevant  to  the 
criteria  established  for  evaluating  this  field 
experiment. 


Relative  Military  Pay 

Recent  models  of  enlistment  supply  have 
incorporated  the  effect  of  relative  military  pay  on 
enlistments.  These  models  were  developed  to 
account  for  large  changes  in  military  pay  and 
benefits  over  time.  Military  pay  does  not, 
however,  vary  greatly  across  markets  during  a 
single  year.  Hence,  most  variation  in  relative 
military  pay  depends  on  differences  in  civilian 
incomes  across  markets.  These  differences  are 
highly  correlated  with  differences  in  unemploy¬ 
ment  rates,  degrees  of  urbanization  and  racial 
composition  across  markets.  The  models  used  in 
this  study  incorporated  these  variables  and  did 
account  for  or  absorb  the  relative  income  effects. 
Models  in  which  measures  of  civilian  income 
have  been  explicitly  included  did  not  yield  signifi¬ 
cantly  different  results  from  those  achieved  in  this 
study. 

The  Model 

Multiplicative  models  were  developed  for  total 
applicants,  contracts  and  for  the  key  segments  of 
each  of  these  measures  of  observed  behavior. 
The  multiplicative  formulation,  which  has  been 
widely  used  in  marketing,  captures  some 
nonlinearity  in  response  as  well  as  possible 
interactions  among  the  independent  variables. 

In  these  models,  XUNEMP,  XURBAN, 
XRACE  and  XRECR  represent,  in  log  form,  per 
capita  or  rate  variables  for  unemployment, 
urbanization,  percent  non-white  and  production 
recruiter  person-months,  respectively.  The 
variables  DB,  DC  and  DD  are  dummy  variables 
representing  Cells  Blue,  Green  and  Red, 
respectively.  The  impact  of  the  control  cell  (Cell 
Yellow)  is  absorbed  into  the  constant.  The 
coefficients  of  these  models  are  interpreted  as 
elasticities. 


Specifically,  the  multiplicative  model  takes  the 
form: 

(Applicant  Rate)  =  bo  (Unemployment  Rate)bi 
(Urbanization  Index  )b2  (Racial  Composition 
Index)1?  (Recruiter  person-months  per 
capita^  e*?08  e1*  00  e1^  DD  eE 

where  bj,  to  t>7  are  coefficients  to  be  estimated;  "e" 
represents  the  base  of  natural  logarithms 
(approximately  2.7183)  and  E  is  the  error. 
Because  this  formulation  is  multiplicative,  the 
effect  of  a  change  in  any  one  term  multiplies 
through  all  other  terms. 

The  effects  of  membership  in  Cell  Blue  are 
captured  by  the  coefficient  b5.  Thus,  when  a  test 
market  belongs  to  Cell  Blue,  DB  is  set  to  1,  and 
the  net  multiplicative  effect  on  the  applicant  rate  is 
through  the  term  ehj.  Conversely,  when  a  test 
market  does  not  belong  to  Cell  Blue,  DB  is  zero; 
hence,  ebs  DB  becomes  e°  which  equals  1,  and 
multiplying  the  expression  by  1  does  not  change 
its  value. 

In  the  models,  the  applicant  rate  was 
measured  by  total  applicants  per  1,000 
population.  In  order  to  preserve  a  common  unit 
of  analysis,  the  recruiter  rate  rather  than  the 
absolute  number  of  total  recruiters  was  used.  The 
advantage  of  this  approach  is  that  the  observations 
are  appropriately  scaled  and  the  error  term  E  is 
minimized.  If  absolute  numbers  were  used 
instead  of  rates,  the  measurement  error  would 
have  wide  disparities  because  of  small  and  large 
markets  (i.e.,  heteroskedasticity). 

In  log  form,  the  multiplicative  model  appears 
as: 

log  (Applicant  Rate) 

=  bo  +  b,  (XUNEMP) 

+ 1>2  (XURBAN) 

+ 1>3  (XRACE) 

+  b4  (XRECR) 

+  bs  DB  +  bg  DC  +  b7  DD  +  E 


Thus,  the  model  becomes  linear  and  additive 
when  expressed  in  natural  logarithms.  Note  that 
DB,  DC  and  DD  do  not  appear  as  logarithms 
since  they  are  expressed  in  the  power  of  "e"  in  the 
multiplicative  specification.  The  coefficents  of  the 
logarithm  terms,  such  as  b4,  are  interpreted  as 
elasticities,  reflecting  the  percentage  change  in, 
for  example,  an  applicant  rate  in  response  to  a  one 
percent  change  in  recruiters  per  capita.  Thus,  the 
multiplicative  specification  yields  coefficients  that 
have  meaningful  interpretations  as  elasticities  for 
policymakers. 

REPORTED  BEHAVIOR  AND  ATTITU- 
DINAL  MEASURES 

The  YATS  responses,  which  form  the  basis 
for  both  the  reported  behavior  and  attitudinal 
measures,  were  analyzed  using  rigorous  statistical 
tests.  These  tests  were  designed  to  identify  the 
specific  instances  where  changes  in  advertising 
were  associated  with  significant  changes  in 
reported  behaviors  and/or  attitudes.  Two 
parametric  tests,  the  difference  of  differences  and 
simultaneous  tests  of  significance,  and  one 
nonparametric  test,  sample  rank  ordering,  were 
used. 

Difference  of  Differences  (two-tailed  "tn 
test) 

The  observations  corresponding  to  Cell 
Yellow  for  1983  and  1984  serve  the  necesssary 
purpose  of  "control."  In  Cell  Yellow,  the 
advertising  level  and  mix  in  1984  were 
deliberately  maintained  at  their  1983  levels.  The 
changes  (or  difference)  from  1983  to  1984  in  each 
of  the  non-control  cells  can  be  compared  to  the 
changes  in  the  control  cell.  The  difference  of 
differences  from  Cell  Yellow  (Control)  to  Cells 
Blue,  Green  and  Red  measures  the  changes  that 
occurred  between  1983  and  1984  with  respect  to 
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the  control  cell.  This  is  termed  the  "difference  of 
differences  estimator. 

The  variance  of  such  an  estimator  is 
computed,  and  a  two-tailed  student  "t"  test  is 
conducted  to  determine  whether  the  "difference  of 
differences' "  estimator  is  significantly  different 
from  zero.  The  level  of  significance  was  set  at 
0.10.  The  details  of  this  procedure  are  described 
in  Appendix  F. 

There  exists  an  alternate  approach  to 
understanding  this  "difference  of  differences" 
estimator  of  advertising  effects.  The  change  in 
the  measures  between  1983  and  1984  for  Cell 
Yellow  is  interpreted  as  a  time  trend  that  may  be 
driven  by  any  number  of  factors  other  than 
changes  in  advertising  budgets.  The  change  in 
other  cells'  measures  between  1983  and  1984  is 
interpreted  as  a  sum  of  this  time  trend  (estimated 
by  the  change  in  Cell  Yellow)  and  an  advertising 
effect.  Therefore,  as  an  example,  the  difference 
between  the  change  in  Cell  Blue  and  the  change  in 
Cell  Yellow  (referred  to  as  the  "difference  of 
differences"  estimator)  measures  the  effect  of 
advertising  in  Cell  Blue.  This  same  line  of 
reasoning  can  be  used  with  respect  to  all  other  cell 
pairs. 

Simultaneous  Tests  of  Significance 

An  alternate  test  of  significance  is  conducted 
to  determine  whether  the  "difference  of 
differences"  estimators  for  Cells  Blue,  Green  and 
Red  are  simultaneously  equal  to  zero.  This  test 
examines  the  hypothesis  that  all  the  1983  and 
1984  observations  came  from  a  homogeneous 
population  with  identical  adverdsing  effects  in  all 
cells.  The  ability  to  reject  this  hypothesis  might 
yield  clues  regarding  which  cells  were  signifi¬ 
cantly  different  and  by  how  much.  The  critical 
0.10  significance  level  is  obtained  from  the  stu- 
dentized  range. 


Sample  Rank  Ordering 

Three  sample  constructions  are  available  and 
are  described  in  the  next  section.  Determining  the 
extent  to  which  the  results  are  sensitive  to  the 
choice  of  sample  is  important.  To  examine  the 
sensitivity  of  the  changes  in  measures  to  the 
sample  construction,  a  nonpar ametric  rank 
ordering  procedure  was  implemented.  A  sample 
rank  ordering  of  the  percentage  changes  in 
measures  with  respect  to  the  1983  levels  was 
made  for  each  of  the  three  sample  constructions. 

Criteria  for  Significance 

Each  of  the  reported  behavior  and  attitudinal 
measures  was  subjected  to  the  parametric  tests 
and  the  sample  rank  ordering.  Therefore,  state¬ 
ments  in  this  report  asserting  that  the  effect  of 
advertising  is  "significant"  for  a  certain  question 
are  based  on  findings  that  met  the  criteria 
established  in  these  parametric  tests  and  that  are 
consistent  in  the  non-parametric  ranking 
procedure. 

ISSUES  FOR  CONSIDERATION 
Surveys  vs.  Direct  Observations 

The  measures  based  on  YATS  responses  are 
not  observed.  They  represent  reported  behavior 
or  attitudes  among  a  sample  of  the  population  of 
interest.  Measures  derived  from  the  YATS 
responses  are,  therefore,  estimates  of  underlying 
behavior  and  attitudes:  Observed  behavior  is 
more  valid. 

Change  in  1984  Protocol 

Changes  in  the  YATS  protocol  between  1983 
and  1984  posed  problems  relating  to  the 


construction  of  comparable  samples.  To  reduce 
sampling  costs,  the  1984  YATS  sample  included 
some  respondents  whose  phone  numbers  were 
called  in  1983.  This  resulted  in  possible  duplicate 
interviews.  A  second  change  in  the  study's 
design  involved  the  questions  concerning  the 
respondents'  intentions  to  join  the  military 
Reserves.  In  1983,  under  the  original  format,  the 
sections  on  active  duty  and  reserve  duty  were 
separated.  Some  questions  administered  to  the 
"Reserve"  sample  were  not  administered  to  the 
"active"  sample  and  vice  versa.  In  1984,  the 
questions  of  interest  were  not  only  asked  in  the 
same  sequence  but  also  asked  of  the  entire  sample 
of  respondents. 

Sample  Selection 

Although  YATS  measures  the  responses  of  a 
wide  range  of  individuals,  including  males  and 
females,  for  the  purposes  of  the  test,  the  YATS 
sample  was  narrowed  to  males  aged  16  to  21 
because  they  constitute  the  primary  target  audi¬ 
ence  for  military  recruiting.  In  1983,  the  total 
number  of  relevant  cases  was  4,415.  To  address 
the  changes  in  the  1984  protocol,  three  sub¬ 
samples  of  the  YATS  data  were  considered: 

Sample  1:  New  interviews  only,  without  re¬ 
serve  respondents  and  without 
respondents  whose  phone  numbers 
were  included  in  the  1983  sample 
(reinterviews)  (2,547  cases). 

Sample  2:  New  interviews  and  "reinterviews," 
without  reserve  respondents  (4,401 
cases). 

Sample  3:  New  interviews  and  "reinterviews," 
with  reserve  respondents  (5,057 
cases). 


Sample  3  was  selected  as  a  base  for  analysis 
for  1984.  Sample  3  contains  the  largest  sample. 
In  addition,  the  questions  on  the  reported 
behavior  and  attitudinal  measures  of  interest  were 
asked  in  the  same  order  to  both  the  "Reserve"  and 
"active"  respondents  in  1984.  Finally,  the  bias 
estimation  and  reduction  techniques  employed  to 
accommodate  the  reinterview  respondents  were 
considered  appropriate. 

A  caveat  on  surveys  in  general  should  be 
given.  Inaccuracies  in  either  the  sampling  scheme 
or  in  the  answers  given  by  respondents  result  in 
measurement  error.  These  inaccuracies  are 
amplified  when  the  smaller  sample  is  scaled  up  to 
the  national  population. 

Quality  of  Respondents 

The  quality  of  the  respondents  whose 
observed  behavior  measures  were  analyzed  can  be 
determined  with  precision  (through  their  AFQT 
group  level).  In  the  YATS  study,  the  quality  of 
individual  respondents  can  only  be  approximated 
because  no  definitive  standardized  test  results  are 
available  from  the  survey  data.  Although  the 
aggregate  samples  could  have  been  subdivided  by 
educational  level,  the  benefits  that  could  be 
achieved  by  such  subdivision  might  have  been 
outweighed  by  the  inaccuracies  in  respondents' 
self-classification.  Therefore,  analysis  was  based 
on  the  aggregate  sample  and  not  on  any  other 
proxy  measure  for  quality. 

Inclusion  of  Other  Explanatory  Factors 

In  addition  to  advertising,  other  factors  would 
seem  to  influence  attitudes  and  behavior  with 
regard  to  military  enlistment.  Factors  such  as 
unemployment,  size  of  the  recruiter  force  or 
urbanization  could  significantly  affect  the  reported 
behavior  and  attitudes  of  the  respondents.  The 


DoD  Advertising  Mix  Test  was  conducted  with 
only  four  cells--too  few  to  estimate  jointly  the 
effects  of  advertising  and  other  influencing  factors 
on  behavior  and  attitudes.  Statistically,  there  are 
insufficient  degrees  of  freedom  to  include  other 
explanatory  variables.  If  the  analysis  were 
conducted  at  the  ADI  level,  these  explanatory 
variables  could  be  included.  This  would  require 
deleting  those  ADIs  with  unacceptably  low 
numbers  of  observations.  Such  an  approach 
would,  however,  raise  the  issue  concerning  the 
definition  of  an  acceptable  number  of 
observations.  To  avoid  these  conflicts,  the 
analysis  was  confined  to  the  cell  level  in  which 
the  effect  of  other  explanatory  variables  on  the 
control  cell  could  be  incorporated  into  the 
"difference  of  differences"  estimator. 

Choice  of  Questions 

Several  questions  in  the  YATS  aim  to  measure 
attitudes  about  military  service.  The  analysis  is 
confined  to  those  questions  devoted  to  measuring 
the  effect  of  advertising  towards  military  service. 
More  specifically,  the  analysis  considers  those 
proven  by  previous  research  to  demonstrate  a 
direct  and  positive  relationship  to  future  enlist¬ 
ment. 

Weighting  Procedure 

A  weighting  procedure  to  adjust  the  YATS 
survey  responses  to  reflect  national  proportions 
has  been  documented  (R.E.  Mason,  "Estimation 
Procedures  in  YATS,"  RTI  Technical  Report, 
December  16,1983).  The  weights  are  constructed 
based  on  the  sampling  scheme.  First,  a  procedure 
called  Mitofsky-Waksberg  clustering  technique  is 
used  to  identify  telephone  numbers.  In  this  case, 
clusters  were  determined  by  area  codes  and  the 
first  three  digits  of  the  phone  numbers. 


Then,  households  within  clusters  are  sampled. 
Finally,  eligible  persons  within  a  household  are 
sampled.  The  actual  number  of  respondents  is 
multiplied  by  the  established  weight,  which  yields 
a  figure  representing  the  total  number  of  youth 
(within  the  relevant  age  and  gender  categories)  in 
the  United  States. 

We  desired  to  conduct  our  analysis  of  the 
YATS  data  based  on  numbers  that  reflect  national 
population  proportions  in  each  cell.  Using  this 
weighting  procedure  to  scale  sample  responses  to 
the  national  level  preserves  the  average  response. 
(The  average  response  is  measured  by  the 
proportion  of  cases  which  responded  favorably  to 
a  particular  question.)  Such  scaling  may, 
however,  substantially  affect  the  variance  of  the 
response  measure.  The  national-level  population 
projections  (derived  using  the  YATS  weights) 
were  scaled  down  so  that  the  total  number  of 
responses  in  all  cells  equaled  the  original  sample 
size  in  the  cells  (as  defined  in  the  DoD 
Advertising  Mix  Test).  The  adjusted  cell-level 
sample  reflecting  national  population  proportions 
was  computed  by  first  calculating  the  ratio  of  the 
total  original  (i.e.,  unweighted)  sample  size  to  the 
total  weighted  sample  size  across  all  cells,  and 
then  the  total  weighted  total  for  each  cell  was 
multiplied  by  this  ratio  to  obtain  the  adjusted  cell- 
level  sample. 

SUMMARY 

This  chapter  described  several  analytical 
approaches  and  quantitative  techniques  used  to 
evaluate  the  data  collected  from  the  field 
experiment.  The  methods  chosen  resulted  from 
several  factors,  including  the  truncated  design  of 
the  experiment  (i.e.,  a  partial  instead  of  a  full 
factorial  design),  the  kinds  of  variables  developed 
from  the  data  collected  (such  as  ratio  and 
categorical),  and  the  efforts  to  cross-validate  the 


findings  using  one  approach  with  that  of  another. 

To  evaluate  advertising's  effect  (if  any)  on 
those  measures  representing  observed  enlistment 
behavior,  a  multiplicative  log  formulation  of  a 
model  was  constructed.  The  coefficients  fitted  to 
the  model  were  derived  from  cross-sectional 


regression  techniques  using  ADIs  as  the  geo¬ 
graphic  unit  of  analysis. 

To  evaluate  advertising's  effect  on  measures 
representing  reported  behavior  and  attitudes 
toward  military  service,  a  series  of  parametric  and 
non-parametric  statistical  tests  were  undertaken. 
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CHAPTER  6 
RESULTS 


Previous  chapters  have  described  the  study 
design,  the  analytical  approach  and  the  measures 
used  to  evaluate  advertising  effectiveness.  The 
first  section  of  this  chapter  summarizes  the  results 
of  the  DoD  Advertising  Mix  Test  and  its 
implications.  The  subsequent  sections  provide 
the  findings  with  respect  to  the  three  measures  of 
observed  behavior,  reported  behavior  and 
altitudinal  responses,  respectively,  (described  in 
Chapter  4).  Additional  interpretations  of  these 
results  are  discussed  in  Chapter  7. 

SUMMARY  OF  RESULTS  AND 
IMPLICATIONS 


TEST  CELL  DESIGN 


Media  Budget 
(Millions  of  $) 

Yellow 

(Control! 

Green 

Blue 

Red 

Service-specific 

68 

68 

15 

15 

Joint 

16 

4 

16 

40 

•  Cell  Blue  performed  better  than  or  at  least  as 
well  as  the  other  cells.  This  result  is 
consistent  for  all  three  measures. 


•  Recruitment  performance  during  the  test  year 
in  Cell  Blue  (which  had  the  lowest  total 
advertising  expenditure)  was  not  adversely 
affected.  In  fact,  the  evidence  suggests  that 
recruiting  system  performance  actually 
increased  at  Cell  Blue's  level  and  mix  of 
advertising  expenditures. 

One  direct  implication  from  these  results  is  the 
implementation  of  Cell  Blue's  budget  levels  on  a 
national  basis.  The  budget  would  need  to  be 
adjusted  for  inflation  in  media  costs  and  could  be 


tested  for  one  year.  Tracking  and  evaluating 
market  responses  to  this  advertising  mix  and 
spending  level  would  be  recommended.  In 
addition,  alternative  spending  levels  should  be 
maintained  in  some  markets.  Longer-term 
(beyond  one  year)  effects  of  Cell  Blue's  budget 
levels  are  not  known. 

Alternate  implications,  based  on  further 
analysis  and  interpretation  of  the  observed 
behavior  results,  are  reviewed  in  Chapter  7. 

RESULTS 

Three  measures  were  used  to  evaluate  adver¬ 
tising  effectiveness:  observed  behavior,  reported 
behavior,  and  attitudes.  Having  validated  these 
measures  and  identified  their  sources  (Chapter  4), 
we  shall  now  review  how  various  levels  of  adver¬ 
tising  expenditures  affected  these  three  measures. 

Impact  of  Advertising  Treatment  on 
Observed  Measures  of  Behavior 

Three  measures  of  observed  behavior  were 
used  in  the  analysis: 

•  Number  of  applicants  taking  the  enlistment 
test 

•  Number  of  contracts  signed 

•  Applicant-to-contract  conversion  ratio,  i.e., 
the  number  of  applications  required  per  con¬ 
tract.  Advertising  may  increase  applicants 
without  a  corresponding  increase  in  contracts, 
leading  to  inefficiency.  Conversely,  advertis¬ 
ing  may  increase  the  yield  of  a  given  applicant 
pool,  thus  enhancing  system  efficiency. 
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Each  of  these  measures  was  analyzed  at  two 
levels: 


Total  DoD  level:  Does  the  number  of 
enlistment  contracts  for  all  Services  change 
from  one  cell  to  another? 


Individual  Service  share  level:  For  example, 
does  the  Army's  share  of  total  applicants 
change  from  one  cell  to  another? 


The  following  key  segments  of  male,  non¬ 
prior  service  (NPS)  applicants  or  enlistees  were 
also  analyzed: 


High  school  graduates  and  seniors 

Non-high  school  graduates  or  seniors 

High  school  graduates  or  seniors  who  were 

also  AFQT  Category  I-IHA 

High  school  graduates  or  seniors  who  are  also 

AFQT  Category  I-in 


Overview  of  Results  from  Observed 
Behavior  Measures 

The  analysis  supports  the  hypothesis  that  the 
advertising  treatments  produced  significant 
differences  in  the  observed  behavior  measures 
(applicants  and  contracts).  In  particular: 


Cell  Blue  generally  did  better,  never  worse, 
than  the  control  (Cell  Yellow)  at  a  high  level 
of  statistical  significance. 


•  Cell  Green  never  did  better  than  the  control 
and  often  performed  worse  at  a  high  level  of 
significance. 


For  the  high  school  graduate  and  senior 
segment,  Cell  Blue  did  better  than  the  control. 
In  contrast.  Cell  Green  provided  worse  results 
than  the  control  for  both  applicants  and 


contracts. 


The  hypothesis  that  advertising  treatment  can 
create  significant  differences  in  Service  share  of 
observed  behavior  measures  has  little  support. 
More  specifically,  the  total  level  of  DoD 
advertising  treatment  budget  or  its  mix  between 
Service-specific  and  Joint  does  not,  generally, 
affect  individual  Service  shares  of  contracts  or 
applicants  (given  that  individual  Service  shares  of 
the  Service-specific  budget  components  are 
maintained).  An  exception  to  this  observation  is 
found  for  the  Marine  Corps  share  of  high  school 
applicants  (see  Table  6.4). 


DOD  Level  Analysis 


Data  Pooled  Across  Cells  (T able  6.1) 


Applicants.  The  model  for  total  applicants  is 
specified  in  Chapter  5. 

With  the  exception  of  the  non-high  school 
group,  the  models  for  applicants  have  generally 
good  explanatory  power.  The  non-advertising 


TEST  CELL  DESIGN 


Media  Budget 
(Millions  of  $) 
Service-specific 
Joint 


Yellow 

(Control) 

68 

16 


Green 
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Independent 

Variable 

Applicant 

Rate 

Contract 

Rate 

Conversion 

Rate 

CONSTANT 

.30 

-.03 

.33 

XRECR 

.61** 

.68** 

-.07* 

XUNEMP 

.32** 

21** 

.05 

XRACE 

.06** 

.02 

.04** 

XURBAN 

.01** 

.01* 

.00 

DB  (Blue) 

.11** 

.09 

.02 

DC  (Green) 

-.10* 

-.09 

-.01 

DD  (Red) 

-.02 

-.003 

-.017 

R2 

55+** 

57*** 

.23 

*  -  t  Statistic  significant  at  0.10  level. 

**  -  t  Statistic  significant  at  0.0S  level. 

***  •  F  Statistic  significant  at  0.01  level. 

N  a  72  observations 


XRECR  denotes  the  natural  logarithm  of  the  recruiter  person-months  per  1,000 
population  in  the  ADI 

XUNEMP  denotes  the  natural  logarithm  of  the  unemployment  rate  in  the  ADI 

XRACE  denotes  the  natural  logarithm  of  the  racial  composition  index  for  the  ADI 

X URBAN  denotes  the  natural  logarithm  of  the  urbanization  index  for  the  ADI 

DB,  DC  and  DD  are  dummy  variables  that  are  set  to  1  if  the  ADI  belongs  to  Cell  Blue,  Green 
or  Red,  respectively,  and  zero  otherwise. 


Reading  the  Table:  If  unemployment  increases  by  100%,  then  applicants  will  increase  by 
32%,  contracts  will  grow  by  27%  and  the  conversion  of  applicants  to 
contracts  (number  of  applicants  per  contract)  will  go  up  by  5%. 


variables  have  the  expected  signs,  and  recruiters 
display  highly  significant  elasticities  for  total 
applicants  and  high  schoolers  of  both  aptitude 
group  segments. 

The  dummy  variable  for  Cell  Blue  is 
significantly  higher  than  control  for  total 
applicants  (at  the  0.05  level)  and  also  for  high 
schoolers.  It  is  higher  than  control  for  the  other 
segments  but  at  lower  levels  of  statistical 
significance.  Cell  Green  is  lower  than  control  (at 
the  0.10  level)  for  high  schoolers,  and  AFQT 
category  IIIB  and  higher. 

Contracts.  The  model  is  defined  as: 

log  (Contract  Rate)  =  bo  +  bj  (XRECR) 

+ 1>2  (XUNEMP) 

+  bj  (XRACE) 

+  b4  DB  +  b3  DC 
+  b$  DD  +  E 

Again,  Cell  Blue  does  significantly  better  than 
control  for  total  contracts  (at  the  0. 10  level).  Cell 
Green  does  significantly  worse  in  three  out  of  five 
models  (see  Table  6.13).  The  models  for  con¬ 
tracts,  as  for  applicants,  are  generally  satisfactory, 
except  for  non-high  school  graduates.  The  poorer 
fit  for  this  group  is  probably  because  of 
constraints  imposed  by  the  Services  concerning 
this  category  of  potential  enlistees. 

Conversion  Rates.  The  models  provide  no 
evidence  of  increased  efficiency  of  the  recruiting 


system  caused  by  advertising  treatments.  Overall, 
the  same  or  greater  numbers  of  applicants  were 
required  to  generate  an  enlistment  contract. 

The  models  estimated  are  of  the  form: 

log  (Conversion  Rate)=  bo  +  bi  (XRECR) 

+ 1>2  (XUNEMP) 

+ 1>3  (XRACE) 

+  b4  (XURBAN) 

+b3  DB  +  bg  DC 
+ 1>7  DD  +  E 

The  models  have  low  R2  values,  indicating 
that  there  is  not  much  variation  in  these  ratio  data 
that  is  explainable  by  the  independent  variables. 
The  only  cell  effect  worth  noting  is  the  strongly 
positive  impact  of  Cell  Green  for  non-high  school 
graduates.  In  other  words,  the  advertising 
treatment  for  Cell  Green  brings  in  a  significantly 
higher  number  of  applicants  per  contract  among 
the  non-high  school  group. 

Conclusions.  Overall,  an  examination  of  the 
annual  cross-sectional  regression  models  for 
applicants,  contracts  and  conversion  ratios, 
reveals  that  Cell  Blue  has  done  better  than  or  as 
well  as  control  in  all  cases.  In  contrast.  Cell 
Green  performed  worse  than  or  as  well  as 
control.  This  is  particularly  noteworthy  for  the 
high  school  segment.  Additional  tables  for  each 
candidate  category  are  presented  at  the  end  of  this 
chapter  (Tables  6.12  to  6.14). 


Media  Budget 
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Service-specific 
Joint 
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Table  6.2 


Regression  Coefficients  (Elasticities)  Reflecting  Changes  in 
Observed  Behavior  Measures 


DoD  Seniors  and  High  School  Diploma  Graduates 
Paired  Cell  Yellow  to  Cell  Blue 


Independent 

Variable 

Applicant 

Rate 

Contract 

Rate 

Conversion 

Rate 

CONSTANT 

54 

.38 

.14 

XRECR 

.69 

.80** 

-.11 

XUNEMP 

28** 

23** 

.06 

XRACE 

.04 

.00 

.04** 

XURBAN 

.09 

.01 

.08 

DB  (Blue) 

.12** 

.09 

.03 

R2 

.46*** 

60*** 

20 

N»  43  observations 


*  •  t  Statistic  significant  at  0.10  level. 
**  -  t  Statistic  significant  at  0.05  level. 
***  •  F  Statistic  significant  at  0.01  level. 


Cross-Sectional  Analyses  with  Cell  Pair  Data. 
The  cross-sectional  models  using  all  the  ADIs 
have  been  supplemented  by  models  based  on 
ADIs  from  pairs  of  test  cells.  These  models  test 
the  hypothesis  that  the  cells  in  the  pair  exhibit 
different  responses  because  of  their  respective 
advertising  treatments.  Of  particular  importance 
is  the  comparison  of  each  test  cell  with  the  control 
cell.  These  "pair-wise"  analyses  have  been  per¬ 
formed  using  1984  data  alone  and  using  changes 
from  1983  to  1984. 

For  each  data  set,  multiplicative  models  were 
developed  for  the  various  applicants  and  contracts 


segments  and  their  respective  conversion  ratios. 
In  these  models,  only  one  dummy  variable  was 
used  to  capture  the  difference  in  the  response  of 
the  cell  pairs  being  examined.  The  cell  pair  of 
greatest  interest  is  Cell  Yellow-Cell  Blue.  The 
elasticities  estimated  for  seniors  and  high  school 
diploma  graduates  (HSDG)  are  displayed  in  Table 
6.2  for  applicants,  contracts  and  conversion 
ratios,  respectively.  Additional  tables  for  other 
candidate  categories  are  in  Tables  6.15  to  6.17  at 
the  end  of  this  chapter.  Models  were,  of  course, 
developed  for  the  other  cell  pairs  as  well. 
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Applicants.  For  applicants,  the  models  are  good 
except  for  non-high  school  graduates.  The  form 
of  the  models  estimated  is: 

log  (Applicant  Rate)  =  bo  +  bi  (XRECR) 

+  bj  (XUNEMP) 

+  bj  (XRACE) 

+  b4  (XURBAN) 

+  bj  DB  +  E 

The  dummy  variable  DB  (Cell  Blue)  is 
significantly  different  from  zero,  with  strong 
effects  for  total  applicants,  high  school  graduates 
and  seniors,  and  high  quality  (AFQT I-IHA)  high 
school  graduate  rates. 

Contracts  and  Conversion  Rates.  The  results  for 
contracts  are  similar,  with  strong  treatment  effects 
displayed  for  the  key  segments  of  high  school 
graduates  and  seniors,  and  high  quality  HSDGs. 
Models  for  conversion  ratios  are  again  poor,  with 
no  significant  treatment  effect  The  models 
estimated  are: 

log  (Contract  Rate)  »  bo  +  bj  (XRECR) 

+  b2  (XUNEMP) 

+  b3  (XRACE) 

+  b4  (XURBAN) 

+  bj  DB  +  E 

log  (Conversion  Rate)*  bo  +  bj  (XRECR) 

+  1)2  (XUNEMP) 

+ 1>3  (XRACE) 

+  b4  (XURBAN) 

+  bj  DB  +  E 


If  instead  of  a  dummy  variable,  the  actual  per 
capita  expenditure  for  Service-specific  advertising 
in  each  of  the  ADIs  in  Cell  Yellow  and  Cell  Blue 
were  used  (with  Joint  being  relatively  constant), 
we  would  expect  and  did  find,  a  negative 
elasticity  for  these  variables.  Statistically  signifi¬ 
cant  negative  elasticities  were  recorded  specifi¬ 
cally  for  the  high  school  segment  with  regard  to 
both  applicants  and  contracts. 

Changes  in  Applicant  Rates.  A  final  set  of 
models  was  developed,  again  on  cell  pairs,  where 
the  variables  were  defined  to  be  changes  between 
1983  and  1984.  Since  racial  composition  and 
urbanization  are  not  expected  to  change  signifi¬ 
cantly  in  the  period  of  a  year,  these  variables  were 
dropped,  and  a  multiplicative  model  was  devel¬ 
oped  relating  changes  in  the  observed  behavior 
measure  to  changes  in  recruiter  effort,  unemploy¬ 
ment  and  to  a  dummy  variable  (representing  the 
difference  in  treatment  effects  between  die  cell 
pairs).  The  models  for  the  pair  Cell  Yellow-Cell 
Blue  are  shown  in  Table  6.3  for  applicants.  ___ 
The  models  are  all  very  poor  with  low  R2 
values.  This  is  to  be  expected  in  cross-sectional 
change  models.  However,  the  dummy  variable 
DB  (Cell  Blue)  is  positive  and  significant  for  total 
applicants,  for  high  school  graduates  and  for  high 
quality  I-IIIA  high  school  graduates.  The  find¬ 
ings  are  similar  for  contracts  and  conversion 
ratios. 

Conclusions.  As  indicated  in  the  analysis  for  data 
pooled  across  cells,  the  models  indicate  that  Cell 
Blue  has  done  better  than  or  as  well  as  control. 
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Table  63 

Regression  Coefficients  (Elasticities)  Reflecting  Changes 
in  DoD  Applicant  Rates  by  Category 


Independent 

Variable 

Total 

Applicants 

HSDG+ 

Seniors 

I-IIIA 

HSDG 

NHS 

CONSTANT 

-.01** 

l 

o 

»-* 

• 

• 

.00 

.00 

-.002** 

XRECR 

.19 

.01 

.26 

.18 

.01 

XUNEMP 

-.03 

-.05 

-.07 

.03 

.00 

DB  (Blue) 

.03* 

.002* 

.002* 

.00 

.00 

R2 

.01 

.02 

.00 

.00 

-.06 

Note:  Coefficients  (Elasticities)  were  derived  from  changes  in  applicant  rates  between  FY  1983  and  1984  for  pairs 
of  markets  (ADI’s)  within  Cells  Yellow  (Control)  and  Blue. 


N  «  43  observations 


*  -  t  Statistic  significant  at  0.10  level. 
**  -  t  Statistic  significant  at  0.05  level. 


Summary  of  DoD  Level  Results.  The  analysis 
supports  the  hypothesis  in  that  advertising 
treatment  created  significant  differences  in  the 
observed  behavior  measures  (applicants  and 
contracts).  Of  particular  importance  are  the 
following  findings: 

•  Cell  Blue  performed  at  least  as  well  as  the 
control  cell  (Cell  Yellow).  It  usually  did 
better  at  a  high  level  of  statistical  significance. 

•  The  performance  of  Cell  Green  was  often 
worse  at  a  high  level  of  significance  than  the 
control.  It  never  did  better  than  Cell  Yellow. 


•  For  the  high  school  and  senior  segment.  Cell 
Blue  out  performed  the  control  for  both 
applicants  and  contracts.  In  contrast.  Cell 
Green  did  worse  than  the  control  for  those 
same  measures. 

Service  Share  Analysis 

The  results  discussed  so  far  have  examined 
the  effect  of  advertising  on  applicants,  contracts 
and  conversion  ratios  at  the  aggregate  DoD  level. 
Next  to  be  investigated  is  the  degree  to  which 
advertising  treatments  affected  each  Service 
individually.  The  aim  is  to  determine  the  degree 
to  which  advertising  treatments  can  shift  the 
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Table  6.4 


Regression  Coefficients  (Elasticities)  Reflecting 
Changes  in  Service  Share  of  Applicants 


Seniors  and  High  School  Diploma  Graduates 


Independent 

Variable 

Army 

Navy 

Air  Force 

Marine  Corps 

CONSTANT 

-.64** 

-1.67** 

-.66 

-3.38** 

SREC 

.007 

-.009 

.11 

-.11 

XUNEMP 

.08* 

-.09 

.06 

-.06 

XRACE 

-.007 

.02 

-.003 

.003 

XURBAN 

-.003 

.01** 

.0006 

.009 

DB  (Blue) 

.01 

-.02 

.05 

.21* 

DC  (Green) 

-.06 

-.06 

.09 

.05 

DD  (Red) 

-.02 

-.008 

.06 

-.03 

R2 

.05 

.06 

-.01 

-.01 

*  •  t  Statistic  significant  at  0.10  level. 
**  -  t  Statistic  significant  at  0.0S  level. 
•**  •  F  statistic  significant  at  0.01  level. 

N  s  72  observations 


outcome  from  Service  to  Service.  The  Army  has 
the  bulk  of  the  Service-specific  advertising  budget 
and  the  highest  share  of  applicants  and  contracts. 
As  a  result,  the  impact  of  advertising  changes  on 
this  group  bears  special  attention. 

Cross-Sectional  Pooled  Data:  Model  1.  A  series 
of  cross-sectional  models  was  developed  for  the 
observed  behavior  response  measures  (applicants, 
contracts)  and  their  key  segments.  The  share  of 
an  individual  Service  of  the  total  DoD  response 
was  the  dependent  variable.  Independent 
variables  were  the  natural  logarithm  of  Services' 


share  of  recruiter  person-months  in  each  market 
over  the  period  (this  new  variable  is  denoted  by 
SREQ,  exogenous  market  variables  (unemploy¬ 
ment,  percent  non-whites,  percent  urbanization) 
and  treatment  dummy  variables.  Again  multi¬ 
plicative  models  were  developed  to  accommodate 
non-linearities  and  possible  interactions.  The 
explicit  model  specification  is: 

log  (Service  applicants/Total  applicants) 

■  bo  +  bt  (XUNEMP)  +  (XURBAN) 

+  b3  (XRACE)  +  b4  (SREC)  +  b5  DB 
+  bg  DC  +  bj  DD  +  E 
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Table  6.5 

Regression  Coefficients  (Elasticities) 
Reflecting  Service  Changes  in  Share  of  Applicants 

Seniors  and  High  School  Diploma  Graduates 
Paired  Cell  Yellow  to  Blue 


Independent 

Variable 

Army 

Navy 

Air  Force 

Marine  Corps 

CONSTANT 

-.36* 

-1.66** 

-.89 

-.3.41* 

XUNEMP 

.07 

-.05 

-.07 

-.23 

XURBAN 

-.008 

.02** 

-.003 

.02 

XRACE 

.02 

-.03 

-.01 

.03 

SREC 

.35** 

.02 

.15 

-.13 

DB  (Blue) 

.30 

.01 

.05 

-.24* 

R2 

.05 

.06 

-.01 

-.01 

N  »  43  observations 


*  -  t  Statistic  significant  at  0.10  level 
••  *  t  Statistic  significant  at  0.05  level. 


Similar  to  the  total  DoD  level  analysis,  these 
share  models  were  estimated  for  all  observations 
pooled  together  and  for  data  sets  restricted  to  pairs 
of  test  cells.  Table  6.4  presents  the  results,  for 
HSDGs  and  seniors,  of  these  models  for  each 
Service  using  the  pooled  data  (see  Tables  6.18  to 
6.21  for  details  on  other  applicant  categories).  On 
the  whole,  the  models  fit  quite  poorly.  This 
indicates  that  the  shares  of  applicants  among  the 
Services  may  depend  on  other  factors.  Only  in 
the  Marine  Corps  did  an  advertising  treatment 
variable  (DB,  DC  or  DD)  show  any  impact  on 
Service  shares  of  applicants.  The  most  note¬ 
worthy  effects  are  observed  in  Cell  Green  where 
the  Army  achieved  a  higher  share  of  AFQT I-IIIA 


HSDGs,  whereas  the  Navy  and  Air  Force 
experienced  changing  shares  of  lower  quality 
applicants. 

Advertising  had  no  significant  effect  on  Cell 
Red’s  share  of  applicants.  The  only  significant 
effect  noticed  was  with  Cell  Blue's  reduced  share 
of  high  school  applicants  to  the  Marines.  Recall 
that  increased  high  school  applications  were 
achieved  across  all  Services  combined  in  Cell 
Blue.  As  a  consequence,  the  Marine  Corps  was 
receiving  a  smaller  slice  of  a  larger  pie.  Cell  Blue 
had  a  favorable  impact  on  the  Army  share  of 
applicants  and  a  positive  impact  on  total  DoD 
applicants. 
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Model  2  -  Cell  Pair  Data.  Table  6.5  presents  the 
results  of  the  models  for  seniors  and  HSDGs  (see 
Tables  6.22  to  6.25  for  other  applicant  categories) 
developed  on  the  restructured  data  set  of  Cell 
Yellow  and  Cell  Blue. 

The  explicit  form  of  the  model  for  the  Army  is: 

log  (Army  applicants/Total  applicants) 

=  bo  +  bi  (XUNEMP) 

+ 1>2  (XURBAN) 

+ 1>3  (XRACE) 

+  b4(SREQ 
+  bj  DB  +  E 

Under  this  formulation,  model  fit  improves 
somewhat  for  the  Army,  but  not  for  the  other 
Services.  Again,  however,  the  advertising 
treatment  effects  are  quite  limited.  No  influences 
of  advertising  treatment  is  observed  for  Army, 
Navy  or  Air  Force  shares  of  applicants  of  any 
type  (between  Cell  Yellow  and  Cell  Blue). 
Marine  Corps  shares  of  several  subcategories  of 
applicants  (high  school  graduates  and  seniors, 
high  school  AFQT  Category  mB  and  above) 
decline  at  the  0.10  level  of  significance. 

Summary  of  Service  Share  Analysis.  On  the 
whole,  the  analyses  demonstrate  that  the  adver¬ 
tising  treatments  had  little  or  no  effect  on  Service 
shares  of  applicants.  None  of  the  models 
developed  provided  good  fits  to  the  data.  Since 
Cell  Blue  generally  had  a  positive  impact  on  total 


DoD  applicants,  and  no  negative  influence  on 
Army  shares,  this  implies  that  Cell  Blue  had  a 
favorable  or  neutral  impact  on  Army  applicants. 

Individual  Service  proportions  of  the  overall 
budget  for  Service-specific  treatment  advertising 
were  kept  constant  across  treatment  conditions. 
This  allowed  an  investigation  of  the  effect  of  the 
level  of  DoD  advertising  expenditures  and  of  the 
mix  between  Service-specific  and  Joint  adver¬ 
tising  on  individual  Service  "market  shares." 
This  was  done  in  a  manner  analogous  to  the 
analysis  of  these  effects  on  the  DoD  level  of 
response. 

Since  Service  shares  must  sum  to  100,  the 
models  estimated  for  each  Service  are  clearly  not 
independent  However,  models,  similar  to  the 
one  used  for  the  Army,  have  been  estimated  for 
each  of  the  Services.  The  model  coefficients 
(e.g.,  the  elasticity  of  recruiter  effort)  are  not 
independent  estimates  across  Services  and, 
strictly  speaking,  are  not  comparable. 

Many  of  these  problems  inherent  to  a  share 
formulation  are  removed  if  applicant  rates  or 
contract  rates  are  used  as  dependent  variables  (in  a 
way  similar  to  that  used  in  the  total  DoD  level 
models).  However,  these  rate  models  are  much 
harder  to  formulate  and  interpret  than  are  the  share 
models.  Indeed,  the  competitive,  cooperative  and 
interactive  factors  of  one  Service  impacting  the 
performance  of  another  Service  must  be  included. 
Therefore,  share  models  are  presented  here. 
Results  obtained  with  the  rate  models  are 
qualitatively  similar. 
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Media  Budget 

Yellow  Green 

Blue 
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(Millions  of  $) 
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Table  6.6 


Percent  Chance  in  Reported  Behavior  Measures 
Active  and  Reserve  Respondents  1983*1984 


Cell  Yellow 
(Control) 

Cell  Blue 

Cell  Green 

Cell  Red 

Reported  Conversations 
with  Parents 

-32.0  123 

-31.1 123 

-33.5 123 

-25.3 113 

Reported  Recruiter 
Contact  (DoD) 

-16.1  ^ 

-7.4® 

-15.4  S3 

•13.9 

O  Significantly  different  from  control  (yellow). 
I  I  Significantly  different  over  time. 


*  Significant  at  0. 10  level. 
**  Significant  at  0.05  level. 


The  Impact  of  Advertising  Treatment  on 
Reported  Measures  of  Behavior 

The  previous  section  detailed  the  results  of  the 
DoD  Advertising  Mix  Test  on  observed  enlistment 
behavior  (applicants,  contracts).  This  section 
analyzes  the  effects  of  the  test  upon  the  proportion 
of  respondents  reporting  contact  with  a  military 
recruiter  and  the  proportion  of  respondents  who 
report  having  discussions  with  their  parents  about 
the  possibility  of  serving  in  the  military. 

Overview  of  Results  for  the  Reported 
Behavior  Measures 

Both  reported  recruiter  contact  and  reported 
conversations  with  parents  about  enlistment 
decreased  in  all  cells  in  the  Fall  1984  (post 


experiment)  wave  of  YATS  compared  to  the  Fall 
1983  (pre-experiment)  wave.  For  reported 
conversations  with  parents,  the  decreases  in  Cells 
Green,  Red  and  Blue  were  not  significantly 
different  than  the  decrease  in  control  Cell  Yellow 
(Table  6.6).  Hence,  advertising  treatment  did  not 
affect  the  reported  level  of  conversations  with 
parents  about  enlistment 

In  regard  to  reported  recruiter  contact,  for  all 
Services  combined,  the  decrease  in  Cell  Blue  was 
significantly  less  than  the  decreases  in  control  Cell 
Yellow  at  the  0. 10  level.  The  decreases  in  Cells 
Green  and  Red  were  not  significantly  different 
than  the  control  decrease.  Cell  Blue  performed 
better  than  both  the  control  and  the  other  cells  for 
reported  levels  of  recruiter  contact,  for  all 
Services  combined. 
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At  the  individual  Service  level  of  analysis,  the 
effects  of  advertising  treatment  on  reported 
recruiter  contact  was  also  observed.  Service 
shares  of  reported  recruiter  contact  were  defined 
as  the  proportion  of  individuals  reporting  recruiter 
contact  with  any  Service.  Since  an  individual  can 
report  contact  with  recruiters  from  more  than  one 
Service,  these  shares  are  not  constrained  to  add  to 
100.  In  the  control  cell,  these  shares  did  not 
change  significantly  between  1983  and  1984. 

A  look  at  the  results  depicted  in  Table  6.8 
indicates  that  the  advertising  treatment  did  not 
change  the  Army's  share  of  recruiter  contact. 
However,  the  other  Services  experienced 
significant  changes  with  regard  to  their  share  of 
recruiter  contacts: 

•  A  decrease  for  the  Air  Force  in  Cells  Red, 
Green  and  Blue  compared  with  control 

•  An  increase  for  the  Marine  Corps  in  Cells 
Green  and  Red  compared  with  control 

•  A  decrease  for  the  Navy  in  Cell  Red  compared 
with  control 

Detailed  Findings:  Conversations  with 
Parents 

Referring  to  Table  6.6,  though  the  differences 
in  reported  conversations  with  parents  are  signifi¬ 
cant  over  time,  no  cell  shows  a  statistically 
significant  difference  from  the  control  difference. 
As  the  results  indicate,  advertising  treatment  had 
no  effect  on  the  reported  level  of  conversations 
with  parents  about  enlistment. 


Detailed  Findings:  Conversations  with 
Recruiters 

With  respect  to  reported  conversations  with 
recruiters,  the  effect  of  advertising,  when 
measured  by  the  difference  of  differences 
estimator  (a  pairwise  parametric  comparison 
discussed  in  Chapter  5),  was  significant  for  Cell 
Blue  as  shown  in  Table  6.6.  The  studentized 
range  test  (a  simultaneous  test  of  differences, 
discussed  in  Chapter  5)  also  suggests  that  Cell 
Blue  generated  significantly  more  conversations 
with  recruiters. 

Table  6.7  shows  that  Cell  Blue  also 
consistently  had  the  highest  rank  ordering  of 
changes  relative  to  control  Cell  Yellow.  Cell  Blue 
produced  more  reported  conversations  with 
military  recruiters,  even  though  it  had  fewer 
recruiters  per  capita  than  the  other  three  cells. 

Service-Specific  Findings  .Conversations 
with  Recruiters  (Reported  Recruiter 
Contact) 

The  results  are  as  follows  (see  Table  6.8): 

*  Conversion  with  a  Recruiter  of  Any  Service- 
As  shown  in  the  first  row  on  Table  6.8,  Cell 
Blue  experienced  a  decline  in  conversations 
with  recruiters  much  less  severe  than  the 
control  cell.  Unlike  the  other  test  cells,  this 
difference  is  also  statistically  significant.  This 
finding  does  serve  to  corroborate  the  findings 
depicted  in  Table  6.7. 


TEST  CELL  DESIGN 


Media  Budget 


Yellow 


Green 


Blue 


Red 


Table  6.7 

Rank  Order  of  Percent  Changes  in  Recruiter  Contact 


Cell  Yellow 

Cell  Blue 

Cell  Green 

Cell  Red 

New  Interviews 

4 

1 

3 

2 

Active  Interviews 

2/3 

1 

4 

2/3 

Active  and  Reserves 
Interviews 

4 

1 

3 

2 

:e:  4  is  equal  to  the  largest  decrease. 

1  is  equal  to  the  smallest  decrease. 


As  examined  through  the  difference  of  differ¬ 
ences  "t"  test,  no  cell  had  an  advertising  effect 
significantly  different  from  zero.  The  effect 
of  advertising  on  the  Army's  share  of  conver¬ 
sations  with  recruiters  was  statistically 
insignificant 


Cell  Red  had  an  advertising  effect  measured 
by  the  difference  of  differences  estimator,  that 
was  significantly  different  from  zero.  This  is 
also  corroborated  by  the  studentized  range 
test  Cell  Red  experienced  a  significant  drop 


Table  6.8 

Percent  Change  in  Reported  Recruiter  Contact 
Between  1983  and  1984 


YATS  Measure 

Cell  Yellow 

Cell  Blue 

Cell  Green 

Cell  Red 

Reported  Recruiter 
Contact  (DoD) 

-16.1^ 

■7.4© 

-15.4^3 

-13.9 

Air  Force  Share 

+18.8 

.7.1H 

m 

-8.5 

-15.4^ 

Army  Share 

-5.6 

-3.9 

-14.7 

+1.7 

Marine  Share 

-7.4 

-4.8 

m 

+12.11 — 1 

+23.8^ 

Navy  Share 

+23.8 

+12.3 

+19.6 

-14.8^ 

iu‘tu  *».  *.|  n.4,1,1  Vi  wi'i.l  >.k  i.i  t 


in  the  Navy's  share  of  conversations  with 
recruiters  when  compared  to  all  cells 
(including  Control  Cell  Yellow).  Other  cells 
were  not  significantly  affected  by  the 
advertising. 


recruiters:  The  effect  of  advertising  on  the  Air 
Force's  share  of  conversations  with  recruiters 
was  statistically  significant  in  all  treatment 
(not  control)  cells  through  the  studendzed 
range  test  All  advertising  treatments  led  to  a 
decreased  share  of  reported  conversations 
with  Air  Force  recruiters. 


recruiters:  Cells  Green  and  Red  had  an  adver¬ 
tising  effect  that  was  significantly  different 
from  zero  as  suggested  by  the  difference  of 
differences  and  the  studendzed  range  tests. 
They  both  reported  an  increased  share  of 
reported  conversations  with  recruiters. 

The  Impact  of  Advertising  Treatment  on 
Reported  Attitudinal  Measures 

This  section  details  the  results  of  the  DoD 
Advertising  Mix  Test  on  reported  attitudinal 
measures,  as  derived  from  the  Youth  Attitude 
Tracking  Study  (YATS).  Attitudinal  measures 
reflect  the  stated  aided  mention  and  degree  of 
certainty  that  respondents  have  about  possible 
military  enlistment.  These  measures  are:  unaided 
mention  of  joining  the  military,  likelihood  of 


joining  the  military,  and  composite  likelihood  of 
joining  the  military.  A  detailed  analysis  of  these 
measures  was  provided  in  Chapter  4. 

Overview  of  Results  for  the  Reported 
Attitudinal  Measures 

The  analysis  does  not  support  the  hypothesis 
that  advertising  treatments  produced  a  significant 
difference  in  attitude  toward  the  military  or 
individual  Services.  When  results  are  compared 
across  the  three  attitudinal  measures,  inconsistent 
and  internally  contradictory  findings  emerge. 
This  implies  that  the  effect  of  advertising  on 
attitudes  is  indeterminate. 

As  with  the  other  measures  investigated,  all 
three  attitudinal  measures  declined  in  all  cells  in 
the  Fall  1984  (post  experiment)  wave  of  YATS 
compared  to  the  Fall  1983  (pre-experiment)  wave. 

There  were  significant  differences  between  the 
decrease  in  a  treatment  cell  and  the  decrease  in 
control  Cell  Yellow  for  two  of  the  three  attitudinal 
measures.  However,  no  treatment  cell  exhibited  a 
consistent  decrease  across  the  three  attitudinal 
measures  compared  to  control  Cell  Yellow.  For 
example,  a  cell  which  decreased  significantly  on 
one  measure  (e.g.,  unaided  mention  of  joining  the 
military)  did  not  decrease  significantly  and  often 
increased  for  the  other  two  measures  when 
compared  to  control.  Moreover,  the  non- 
parametric  rank  orderings  of  percent  change  in 
these  attitudinal  measures  were  not  consistent 
with  respect  to  either  measures  or  samples.  For 
one  attitudinal  measure,  Cell  Red  may  have 
experienced  the  largest  decline  of  all  cells  while 


Media  Budget 
(Millions  of  $) 
Service-specific 
Joint 


TEST  CELL  DESIGN 

Yellow  Green  Blue 

(Control)  _ _ 

68 _ 68 _ 15_ 

16  4  16 


for  another  measure  it  may  have  suffered  the  least 
when  compared  to  the  other  cells  (including 
control).  In  summary,  the  atdtudinal  measures 
did  differ  significantly  with  respect  to  advertising 
treatment.  The  differences  observed,  however, 
were  not  consistent  across  measures  or  samples. 

Defining  a  relationship  between  advertising 
treatment  and  attitudes  toward  the  military 
depends  upon  identifying  which,  if  any,  of  the 
measures  used  are  reliable,  independent  measures 
of  enlistment  intentions  and  subsequent  enlistment 
behavior.  Since  such  identification  is  not 
currently  available,  the  conclusion  is  that  the 
effect  of  advertising  treatment  on  attitudes  toward 
the  military  is  indeterminate. 

Detailed  Findings:  Unaided  Mention  of 
Joining  the  Military 

Table  6.9  presents  the  percent  change  in  this 
and  die  other  two  attitudinal  measures.  The 
change  is  defined  as  the  difference  in  proportions 
of  positive  responses  between  1983  and  1984 
divided  by  the  proportion  of  positive  responses  in 
1983.  Table  6.10  presents  the  rank  ordering  of 
percent  changes  across  the  three  samples 
investigated.  The  difference  of  differences 
estimator  reveals  that  Cell  Blue  has  an  advertising 
effect  that  is  significantly  different  from  zero,  and 
is  negative  when  compared  with  control.  The 
studentized  range  test  also  points  to  the  same 
conclusion.  Cell  Blue  consistently  had  the  lowest 
rank  ordering  of  changes  relative  to  the  control 
cell  across  all  three  samples.  This  indicates  that 
the  conclusions  are  consistent  across  different 
definitions  of  samples. 


Detailed  Findings:  Unaided  and  Aided 
Mention  of  Joining  the  Military 

The  difference  of  differences  over  time  in 
Ceils  Blue,  Green  and  Red  are  significantly 
different  from  zero.  The  studentized  range  test 
also  shows  that  the  advertising  effects  in  the  test 
cells  are  simultaneously  different  from  zero.  The 
sample  rank  ordering  shows  consistency  for  Cells 
Blue  and  Green,  with  identical  ranks  for  samples 
2  and  3.  The  rank  ordering  for  all  cells  is 
identical  for  samples  2  and  3.  Although  the  effect 
of  advertising  on  the  likelihood  of  joining  the 
military  was  not  statistically  significant,  it  is 
notable  that  Cell  Blue  consistently  had  the  highest 
rank  ordering  of  changes  relative  to  Cell  Yellow 
across  all  three  samples.  This  means  that  Cell 
Blue  did  the  best  among  other  cells  across 
different  sampling  schemes  on  this  measure. 

Detailed  Findings:  Composite  Likelihood 
of  Joining  the  Military 

This  measure  was  derived  from  the  question: 
"How  likely  is  it  that  you  will  be  serving  on  active 
duty  in  the  (Service)?"  The  difference  of 
differences  estimator  of  composite  likelihood  of 
joining  the  military  was  statistically  significant  in 
Cell  Red  by  the  criterion  of  the  two-tailed  "t"  test 
Cell  Red  performed  worse  than  the  control  cell 
and  consistently  had  the  lowest  rank  ordering 
across  all  cells.  Such  differences  in  other  cells 
were  not  significant.  However,  the  studentized 
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TEST  CELL  DESIGN 


Media  Budget 
(Millions  of  $) 

Yellow 

(Control) 

Green 

Blue 

Red 

Service-specific 

68 

68 

15 

15 

Joint 

16 

4 

16 

40 

62 


Table  6.9 


Percent  Change  in  Attitudinal  Measures 
Active  and  Reserve  Respondents 


Measures 

Cell  Yellow 

Cell  Blue 

Cell  Green 

Cell  Red 

Unaided  Mention  of 
Joining  Military 

-21.6 

-43.7  © 

-26.7 

-9.3 

Aided  Mention  of 
Joining  Military 

-15.1 

-5.2 123 

-21.4 

-7.6 

Composite  Likelihood 
of  Joining  Military 

-8.3 

-1.4 

-8.0 

-17.5© 

O  Significantly  different  from  control  (yellow). 
I  I  Significantly  different  over  time. 


*  Significant  at  0. 10  level. 
**  Significant  at  0.05  level. 


Table  6.10 

Rank  Order  of  Change  in  Attitudinal  Measures 


Cell  Yellow  Cell  Blue  Cell  Green 


Cell  Red 


Sample  #1  New  Interviews 
#2  Active  Interviews 

#3  Active  &  Reserve 
Interviews 

Aided  Military 
Sample  #1  New  Interviews 
#2  Active  Interviews 

#3  Active  &  Reserve 
Interviews 


Sample  #1  New  Interviews 
#2  Active  Interviews 

#3  Active  &  Reserve 
Interviews 


n 


Cell  Yellow 

Cell  Blue 

Cell  Green 

Cell  Red 

Air  Force 

-6.1 

-15.3 

-6.5 

+2.0 

Army 

+8.0 

-14.9® 

+17.4 

■5.6© 

© 

Marine  Corps 

-1.5 

-28.3® 

-10.1 

-0.6 

Navy 

+8.1 

+26.6 

© 

-27.9® 

-5.0 

O  Significantly  different  from  control  (yellow). 
1  I  Significantly  different  over  time. 


*  Significant  at  0.10  level. 
**  Significant  at  0.05  level. 


range  test  suggests  that  this  advertising  effect  is 
indistinguishable  from  zero  when  compared 
simultaneously  with  other  differences.  Hence, 
results  for  the  composite  likelihood  measure  are 
not  significant. 

Detailed  Findings:  Service  Share  -  Com - 
posite  Likelihood  of  Joining  the  Military 

Individual  Service  shares  of  the  Composite 
Likelihood  of  Joining  the  Military  measure  were 
computed.  Those  shares  can  sum  to  more  than 
100  percent  since  individual  respondents  can 
indicate  a  definite  or  probable  intention  to  join 


more  than  one  Service.  The  results  of  this 
analysis  are  presented  in  Table  6.11  and  aye 
summarized  as  follows: 


difference  of  differences  estimator  reveals  that 
Cell  Blue  had  advertising  effects  that  were 
significantly  different  from  zero.  The  effect 
of  advertising  on  the  Army  share  of  the 
composite  likelihood  of  joining  the  military 
was  statistically  significant  in  Cells  Blue  and 
Red  as  implied  by  both  the  "t"  test  and  the 
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TEST  CELL  DESIGN 


Green 


Service-specific 

68 

Joint 

16 

Blue 

Red 

15 

15 

16 

40 
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studentized  range  test.  The  Anny's  share 
decreased  in  these  cells  and  increased  in  Cells 
Yellow  and  Green. 


Likelihood  of  Joining  the  Military:  The  effect 
of  advertising  on  the  Navy's  share  of  the  com¬ 
posite  likelihood  of  joining  the  military  is 
statistically  significant  in  Cell  Green.  This 
inference  results  from  both  the  "t"  test  and  the 
studentized  range  test  The  difference  of 
differences  estimator  also  confirms  this.  The 
Navy's  share  of  the  composite  likelihood  of 
joining  the  military  was  lower  in  Cell  Green 
than  in  all  other  cells  including  control  Cell 
Yellow. 


difference  of  differences  estimator  reveals  that 
none  of  the  cells  had  an  advertising  effect 
significantly  different  from  zero.  Advertising 
had  a  statistically  insignificant  effect  on  all 
cells.  This  is  implied  by  both  the  "t"  test  and 
the  studentized  range  test.  Thus,  there  is  no 
statistically  significant  effect  of  advertising  on 
Air  Force  share  of  composite  likelihood  of 
joining  the  military. 
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Likelihood  of  Joining  the  Military:  The  effect 
of  advertising  on  the  Marine  Corps  share  of 
the  composite  likelihood  of  joining  the 


military  is  statistically  significant  in  Cell  Blue. 
The  change  in  the  Marine  Corps  share  of  this 
measure  is  significantly  lower  in  Cell  Blue 
than  the  other  cells. 


The  interpretation  of  these  share  measure 
differences  also  depends  upon  the  validity  and 
reliability  of  the  composite  likelihood  measure 
compared  to  the  other  measures.  This  measure  is 
based  on  repeated  questions  about  a  respondent's 
likelihood  of  joining  specific  Services.  Thus, 
Service-specific  preferences  of  respondents  might 
influence  this  measure. 


DIRECT  IMPLICATIONS  OF 
FINDINGS 


Clearly,  the  results  of  the  DoD  Advertising 
Mix  Test  have  implications  for  DoD  advertising 
budget  policy.  After  summarizing  the  results 
across  measures,  budget  implications  based  only 
on  cell  performance  during  the  test  are  discussed 
and  calculated.  These  are  the  direct  implications. 
Alternate  budget  implications,  based  on  additional 
analyses  of  applicant  rates,  are  reviewed  in 
Chapter  7. 

Assessments  of  the  effects  of  the  level  and 
mix  of  advertising  on  the  enlistment  process  were 
made  by  comparing  the  performance  of  the  mea¬ 
sures  of  the  enlistment  process  in  each  cell  with 
their  performance  in  the  control  cell.  The  context 
of  the  experiment  was  one  in  which  all  measures 
of  recruiting  system  performance  declined  in  the 


TEST  CELL  DESIGN 


Media  Budget 
(Millions  of  $) 
Service- specific 
Joint 


Yellow 

(Control) 

68 

16 


Green 


5 


•s'// 
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test  year  compared  to  the  previous  year. 

Within  this  context,  the  results  of  the 
comparisons  of  the  treatment  cells  (Cells  Blue, 
Green  and  Red)  with  the  control  cell  (Cell 
Yellow)  were  consistent  across  measures,  units  of 
analysis  and  methods  of  analysis.  Cell  Blue 
performed  significantly  better  than  the  control  cell 
(Cell  Yellow)  and  better  than  or  no  worse  than 
Cells  Red  and  Green  when  measured  by  the 
enlistment  contract  rate  per  unit  of  population  and 
by  the  rate  of  applicants  per  unit  of  population. 
These  differences  were  strongly  evident  for  high 
school  graduates  and  seniors,  as  well  as  for 
totals.  It  is  also  noteworthy  that  Cell  Green 
performed  significantly  worse  than  control  for  the 
key  high  school  graduate  and  senior  category. 

No  differences  were  observed  across  cells  in 
the  conversion  rate  from  applicants  to  contracts 
far  any  category  of  recruits.  Finally,  few  sig¬ 
nificant  differences  in  the  market  shares  of 
enlistments  or  applicants  were  observed  for  any 
Service  for  any  category  of  recruits  in  Cell  Blue 
compared  with  the  control  cell.  The  only 
exception  to  this  may  be  the  Marine  Corps  share 
a i  high  school  graduate  and  senior  applicants. 
(Even  here,  little  if  any  negative  impact  on  the 
Mannc  Corps  level  of  such  applicants  is  to  be 
anonpased  because  its  reduced  share  occurs  from 
a  larger  pool  of  applicants.)  Hence,  Cell  Blue 
performed  better  on  these  observed  behavior 
iwanti  with  additional  applicants  evidently 
leading  10  additional  enlistment  contracts  of 


various  categories. 

On  the  reported  behavior  measures.  Cell  Blue 
also  performed  better  than  the  control  cell  for 
reported  recruiter  contact  and  the  same  as  the 
control  cell  for  reported  conversations  with 
parents  about  enlistment  (for  which  no  cell 
differed  significantly  from  the  control  cell)  at  the 
DoD  level.  The  Air  Force  share  of  reported 
recruiter  contact  decreased  significantly  relative  to 
the  control  cell.  Again,  this  decrease  in  share  was 
offset,  at  least  partially,  by  the  higher  level  of 
overall  military  recruiter  contact  reported.  Hence, 
Cell  Blue  performed  better  on  the  reported 
behavior  measures  with  additional  recruiter 
contact  evidently  leading  to  additional  applicants 
and  producing  additional  enlistment  contracts  of 
various  categories. 

Finally,  the  effect  of  advertising  treatment  on 
altitudinal  measures  of  intention  to  join  the 
military  is  indeterminate.  For  example,  in  non- 
parametric  rank  order  comparisons.  Cell  Blue 
performed  the  best  on  two  such  measures  and  the 
worst  on  one.  Significant  additional  analysis  of 
both  historical  response  and  enlistment  data  and 
the  experimental  response  data  is  required  in  order 
to  ascertain  the  effect  of  advertising  treatment  on 
enlistment  intentions  and  subsequent  enlistment 
behavior. 

The  finding  that  Cell  Blue  performed  better 
than  the  control  cell  and  better  than  the  other 
treatment  cells  is  notable  because  Cell  Blue  had 
the  lowest  total  test  advertising  budget  level  of  all 
the  cells. 


TEST  CELL  DESIGN 


Media  Budget 
(Millions  of  $) 

Yellow 

(Control) 

Green 

Blue 

Red 

Service-specific 

68 

68 

15 

15 

joint 

16 

4 

16 

40 

T35S 


N'V.< 


\>V 


•  1  l 


l  "4^* 


V. 


*“  v  . 


A 


•‘.-l 


T 

£ 


jiumj1 1 1 qy  yyyrw n m m u y  j y j y.1 t."1  i" '•". 1  vyjyj yjw: 


Nevertheless,  it  is  clear  that  during  the  one- 
year  period  of  the  test,  the  lower  advertising 
budget  and  spending  level  in  Cell  Blue  did  not 
lead  to  lower  response  levels  for  the  military  as  a 
whole.  Hence,  during  the  one-year  period  of  the 
test,  substantially  reducing  advertising 
expenditures  did  not  lead  to  a  decrease  in  the 
performance  of  the  overall  recruiting  system  given 
that  Service  shares  of  the  reduced  levels  of 
Service-specific  budgets  were  kept  constant.  The 
results  of  the  experiment  hold  for  the  military 
recruiting  system  as  a  whole,  for  the  Army  and 
for  the  other  Services.  These  results  hold  across 
the  various  measures  of  response  investigated  in 
analyses  conducted  and  reviewed  by  us.  In  fact, 
the  recruiting  system's  performance  significandy 
improved  in  Cell  Blue  during  the  one-year  period 
when  other  differences  in  the  cells  were 
considered. 

Hence,  a  direct  implication  of  these  results  is 
to  implement  the  Cell  Blue  budget  levels  on  a 
national  level.  This  implication  is  warranted  on 
the  Hinds  of  both  efficiency  and  effectiveness. 
It  can  be  accompanied  by  the  suggestion  that  the 
market  response  to  the  new  budget  levels  be 
traced  and  tested  further  and  that  some  markets 
be  maintained  at  altemaive  spending  levels  in 
order  to  determine  the  longer  term  effects  of  the 
Cell  Blue  budget  level. 

A  n  umber  of  steps  are  involved  in  determining 
the  l.rect  budgetary  implications  of  the  results  of 
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the  DoD  Advertising  Mix  Test.  First,  budgeting 
guidelines  for  working  media  expenditures  per 
17-21 -year-old  male  per  year  were  derived 
directly  from  the  average  Cell  Blue  expenditure 
level  during  the  year  of  the  test  ($3.96  per  1,000 
17-21-year-old  males).  Guidelines  for  the  Joint 
proportion  of  the  budget  (41  percent)  were 
derived  from  the  average  Cell  Blue  proportion 
during  the  test  ($1.63  Joint  expenditure/$3.96 
total  expenditure  equals  41  percent).  Second, 
these  media  costs  per  17-21 -year-old  male  were 
multiplied  by  the  number  of  17-21 -year-old  males 
in  the  United  States  in  1984.  Non-media 
production  and  labor  costs  incurred  by  each 
advertiser  and  identified  and  included  in  their 
planning  for  the  experiment  were  added  back  to 
the  media  costs.  This  yielded  budget  levels  for 
FY  1984  that  the  experiment's  results  indicate 
would  have  been  more  effective  for  NPS, 
enlisted,  active  force  advertising  budgets.  At  this 
point,  other  advertising  budget  elements  such  as 
reserve  and  officer  advertising,  which  were 
excluded  from  the  test,  were  re-added  to  arrive  at 
the  total  DoD  advertising  budgets  for  FY  1984 
indicated  by  the  test  results.  Finally,  recom¬ 
mended  DoD  enlisted,  active  force  budget  levels 
for  FY  1987  were  calculated  by  inflating  the  1984 
adjusted  budget  levels  by  10  percent  per  year  to 
allow  for  price  increases  in  advertising  costs. 
Total  recommended  FY  1987  DoD  advertising 
budget  levels  were  then  computed  by  adding  back 
excluded  elements  programmed  for  FY  1987. 


TEST  CELL  DESIGN 


Media  Budget 
(Millions  of  $) 

Yellow 

(Control) 

Green 

Blue 

Red 

Service-specific 

68 

68 

15 

15 

Joint 

16 

4 

16 

40 

67 


! 


\V 

r.A 


n 


0 


Vy 


I 


Independent 

Variable 

TOTAL 

HSDG+ 

Seniors 

I-niA 

HSDG 

NHS 

raB-v 

CONSTANT 

.21 

.30 

-.19** 

-3.23** 

-.30 

XRECR 

.54** 

.61** 

.79** 

.18 

.49** 

XUNEMP 

.33** 

.32** 

.17** 

.35** 

.49** 

XRACE 

.05** 

.06** 

-.05* 

.02 

.18** 

XURBAN 

.01  •* 

.01** 

.02** 

.01 

.004 

DB  (Blue) 

.12** 

.11** 

.10 

.18 

.12 

DC  (Green) 

-.09 

-.10* 

-.06 

-.04 

-.16* 

DD  (Red) 

-.03 

-.02 

-.10 

-.06 

.38 

R2 

.52*** 

.55*** 

.50*** 

.08 

.47*** 

•  .  t  Statistic  significant  at  0.10  level. 
••  -  t  Statistic  significant  at  O.OS  level. 

•  F  Statistic  significant  at  O.Ol  level. 


N  *  72  observations 


Table  6.13 


Relative  Contribution  (Elasticities)  of  the  Regression  Variables 

to  Contract  Levels 


N  *  72  observations 


*  -  t  Statistic  significant  at  0.10  level. 

**  •  t  Statistic  significant  at  O.OS  level. 

-  F  Statistic  significant  at  0.01  level. 


Independent 

Variable 

TOTAL 

HSDG-*- 

Seniors 

i-hia 

HSDG 

NHS 

III  B-V 

CONSTANT 

.05 

-.03 

-.46 

-4.78** 

.67 

XRECR 

.65** 

.68** 

.78** 

.16 

.59** 

XUNEMP 

.26** 

.27** 

.14* 

.13 

.46** 

XRACE 

.02 

.02 

-.04 

.04 

.11** 

XURBAN 

.01** 

.01* 

.01  •* 

.03** 

.07 

DB  (Blue) 

.09 

.09 

.06 

.10 

.10 

DC  (Green) 

-.10* 

-.09 

-.07 

-.25** 

-.14* 

DD(Red) 

-.01 

-.003 

-.07 

-.09 

.07 

R2 

.55*** 

.57*** 

.49*** 

.09 

.46*** 

Table  6.14 


Relative  Contribution  (Elasticities)  of  the  Regression  Variables 
to  Conversion  Rates  of  Applicants  to  Contracts 


Independent 

Variable 

TOTAL 

HSDG+ 

Seniors 

I-IIIA 

HSDG 

NHS 

raB-v 

CONSTANT 

.27 

.27 

.26 

1.54** 

.36 

XRECR 

-1.0** 

-.07* 

-.02 

-.01 

-.10 

XUNEMP 

.07* 

.05 

.03 

.22* 

.03 

XRACE 

.03** 

.04** 

.01 

.02 

.07** 

XURBAN 

.00 

.00 

.06** 

.02 

.00 

DB  (Blue) 

..03 

.03 

.02 

.09 

.02 

DC  (Green) 

-.75 

-.00 

-.02 

.21** 

.02 

DD(Red) 

-.21 

-.02 

-.04 

-.03 

.04 

R2 

.15 

.23 

.06 

.06 

.34*** 

*  -  t  Statistic  significant  at  0.10  level. 
*•  -  t  Statistic  significant  at  0.05  level. 
**•  -  F  Statistic  significant  at  0.01  level. 


N  =  72  observations 


Table  6.15 


DoD  Applicant  Rate 
Paired  Cell  Yellow  to  Cell  Blue 


Independent 

Variable 

TOTAL 

HSDG+ 

Seniors 

I-IIIA 

HSDG 

NHS 

m  b-v 

CONSTANT 

.43 

.54 

-.15 

-3.11** 

-.02 

XRECR 

.63** 

.69** 

.75** 

.23 

.66** 

XUNEMP 

.28** 

.28** 

.27** 

.33* 

.31** 

XRACE 

.03 

.04 

.08** 

.03 

.17** 

XURBAN 

.09 

.09 

.02** 

.02 

.01 

DB  (Blue) 

.13** 

.12** 

.13** 

.21 

.09 

R2 

.46 

.49*** 

.63*** 

.08 

.32 

*  •  t  Statistic  significant  at  0.10  level. 
*•  -  t  Statistic  significant  at  0.05  level. 
*••  -  F  Statistic  significant  at  0.01  level. 

N  a  43  observations 


DoD  Contract  Rate 
Paired  Cell  Yellow  to  Cell  Blue 


Independent 

Variable 

TOTAL 

HSDG+ 

Seniors 

I-fflA 

HSDG 

NHS 

in  B-V 

CONSTANT 

.24 

.38 

-.45 

-4.57** 

.03 

XRECR 

.75** 

.80** 

.77** 

.30 

.85** 

XUNEMP 

.21** 

.23** 

.20** 

.09 

.30** 

XRACE 

-.00 

.00 

-.07 

.04 

.11** 

XURBAN 

.09 

.01 

.01 

.02** 

.01 

DB  (Blue) 

.10* 

.09* 

.10** 

.14* 

.08 

R2 

.60*** 

.60*** 

.62*** 

.15 

.39 

*  -  t  Statistic  significant  at  0.10  level. 
**  -  t  Statistic  significant  at  0.0S  level. 
***  -  F  Statistic  signiilcant  at  0.01  level. 

N  =  43  observations 


Table  6.17 

DoD  Conversion  Rate  of  Applicants 
to  Contracts  Paired  Cell  Yellow  to  Cell  Blue 


Independent 

Variable 

TOTAL 

HSDG+ 

Seniors 

I-niA 

HSDG 

NHS 

in  B-V 

CONSTANT 

.19 

.14 

-.29 

-1.46 

.05 

XRECR 

.12 

.11 

.02 

.05 

.19*** 

XUNEMP 

-.07 

.06 

-.07 

-.24 

.02 

XRACE 

© 

* 

-.04** 

.01 

-.01 

-.07** 

XURBAN 

.00 

-.01 

-.01 

.00 

.03 

DB  (Blue) 

-.03 

-.03 

-.02 

-.07 

-.02 

R2 

.10 

.20 

.04 

-.06 

.32 

*  -  t  Statistic  significant  at  0.10  level. 

**  -  t  Statistic  significant  at  0.05  level. 
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N  ■  72  observations 


Table  6.18 

Army  Share  of  Applicants 
Pooled  Data  from  All  Cells 


Independent 

Variable 

__ .  _  HSDG+ 

TOTAL  Seniors 

I-fflA 

HSDG 

NHS 

IDB-V 

CONSTANT 

-.51 

-.64** 

.95* 

-1.04 

-.46 

SREC 

.01 

.007 

.22** 

-.01 

.03 

XUNEMP 

.91** 

.08* 

.14* 

.06 

.05 

XRACE 

-.01 

-.007 

-.03 

-.07** 

.01 

XURBAN 

-.0009 

-.003 

.01 

-.003 

-.0005 

DB  (Blue) 

.02 

.01 

.06 

.009 

.02 

DC  (Green) 

.03 

-.06 

.11* 

-.02 

-.07 

DD(Red) 

-.03 

-.02 

.02 

-.04 

-.01 

R2 

.02 

.05 

.14 

.28 

-.05 

*  . 

+*  . 

t  Statistic  significant  at  0.10  level 
t  Statistic  significant  at  0.05  level 

N  *  72  observations 

Table  6.19 

Navy  Share  of  Applicants 
Pooled  Data  from  All  Cells 

Independent 

Variable 

HSDG- 

TOTAL  Soikn 

I-II1A 

HSDG 

NHS 

niB-v 

CONSTANT 

-1.50** 

-1.67** 

-.77 

-.88 

-2.99** 

SREC 

-.02 

-.009 

.04 

.14 

-.11 

XUNEMP 

-.13* 

-.09 

-.07 

-.36** 

-.11 

XRACE 

.04 

.02 

.13** 

-.07 

-.05 

XURBAN 

.01* 

.01** 

.003 

.009 

.02** 

DB  (Blue) 

-.02 

-.02 

-.03 

-.10 

-.004 

DC  (Green) 

-.03 

-.06 

-.04 

-.26* 

.01 

DD(Red) 

.003 

-.008 

-.02 

-.03 

-.03 

R2 

.08 

.06 

.26 

.11 

.03 

*  •  t  Statistic  significant  at  0.10  level. 
••  •  t  Statistic  significant  at  0.0S  level. 


Table  6.20 


.»< « ai» 


N  =  72  ohservauons 


Air  Force  Share  of  Applicants 
Pooled  Data  from  All  Cells 


Independent 

Variable 

TOTAL 

HSDG+ 

Seniors 

I -III A 
HSDG 

NHS 

mB-v 

CONSTANT 

-.52 

-.66 

-.37 

-3.93 

-1.37 

SREC 

.13* 

.11 

.10 

.06 

.07 

XUNEMP 

-.08 

.06 

-.01 

-.55** 

-.12 

XRACE 

.004 

-.003 

-.04 

.005 

-.01 

XURBAN 

-.004 

.0006 

-.001 

-.007 

-.001 

DB  (Blue) 

.04 

.05 

.03 

-.16 

.02 

DC  (Green) 

.09 

.09 

-.005 

.22 

.20* 

DD  (Red) 

.08 

.06 

.23 

.17 

.13 

R2 

.03 

-.01 

-.04 

.25 

.002 

t  Statistic  significant  at  0.10  level, 
t  Statistic  significant  at  0.05  level. 


Table  6.21 

Marine  Coma  Share  of  Anolicants 


idc  pendent 
Variable 

TOTAL 

HSDG* 

Sen  tors 

I-IHA 

HSDG 

NHS 

III  B-V 

<  <  JNSTANT 

-3.21** 

-3.38** 

-3.79** 

-4.01  ** 

•3.01** 

'iRI-.C 

-.09 

-.11 

-.09 

-.11 

.13 

XUNEMP 

-.04 

-.06 

-.23 

-.05 

.12 

XRACE 

.0006 

.003 

-.06 

.07 

.05 

XURBAN 

.006 

.009 

.02 

© 

* 

.004 

DB  (Blue) 

-.17 

-.21* 

-.18 

.03 

-.21 

DC  (Green) 

.02 

.05 

.07 

.55** 

.05 

DD(Red) 

-.03 

-.03 

-.01 

-.19 

-.13 

R2 

-.04 

-.01 

.02 

.001 

.0007 

Tin, 


r.v./y;. 


Table  622 


Army  Share  of  Applicants 
Cells  Yellow  and  Blue 


Independent 

Variable 

TCITAI  HSDG+ 

TOTAL  Seniors 

i-niA 

HSDG 

NHS 

III  B-V 

CONSTANT 

-.30 

-.36* 

-.55** 

.13 

-.34 

SREC 

.33** 

.35** 

.42** 

.44* 

.28* 

XUNEMP 

.08 

.07 

.06 

.11 

.06 

XRACE 

.009 

.02 

-.03 

-.02 

.02 

XURBAN 

-.004 

-.008 

-.004 

.01 

-.007 

DB  (Blue) 

.03 

.03 

.02 

.05 

.03 

R2 

.11 

.14 

.29 

.10 

-.007 

•  _ 

t  Statistic  significant  at  0.10  level. 

»•  . 

t  Statistic  significant  at  0.05  level. 

N  *  43  observations 

Table  <L23 

Navy  Share  of  Applicants 

Cells  Yellow  and  Blue 

Independent 

HSDG+ 

I-IIIA 

Variable 

1UIAL  semen 

HSDC 

)  NHS 

III  B-V 

CONSTANT 

-3.12** 

-3.41* 

-3.71** 

-2.36** 

-3.21** 

SREC 

-.12 

-.13 

-.10 

.02 

-.15 

XUNEMP 

-.16 

-.23 

-.36 

.08 

-.13 

XRACE 

.03 

.03 

-.007 

.18 

.03 

XURBAN 

.01 

.02 

.02 

-.03 

.01 

DB  (Blue) 

.19* 

-.24* 

-.20 

.03 

-.25* 

R2 

•05 

.09 

.09 

-.07 

.05 

Ts 


I 


Independent 

Variable 

TOTAL 

HSDG+ 

Seniors 

I-III  A 

HSDG 

III  B-V 

CONSTANT 

-1.13** 

-.89 

-.63 

-1.43 

SREC 

.14 

.15 

.14* 

.14 

XUNEMP 

-.09 

-.07 

-.01 

-.19 

XRACE 

-.009 

-.01 

.02 

-.02 

XURBAN 

.008 

-.003 

-.005 

-.00009 

DB  (Blue) 

.04 

.05 

.03 

.01 

R2 

*01 

-.01 

-.03 

.06 

*  -  t  Statistic  significant  at  0.10  level. 

•*  -  t  Statistic  significant  at  0.05  level. 

N  =  43  observations 

Table  6.25 

Marine  Corps  Share  of  Applicants 
Cells  Yellow  and  Blue 

Independent 

Variable 

TOTAL 

HSDG+ 

Seniors 

I-III  A 
HSDG 

NHS 

III  B-V 

CONSTANT 

-1.77** 

-1.66** 

-.96* 

-2.25** 

-2.14** 

SREC 

-.03 

-.02 

.06 

-.04 

-.08 

XUNEMP 

-.07 

-.05 

-.01 

-.23 

-.04 

XRACE 

-.02 

-.03 

-.04 

-.002 

-.06 

XURBAN 
DB  (Blue) 


.02** 

.01 


.01 


-.01 


-.01 


.04** 

.01 


CHAPTER  7 


ADDITIONAL  ANALYSES 


The  results  of  the  test,  as  indicated  in  Chapter 
6,  clearly  show  that  Cell  Blue  performed  better 
than,  or  at  least  as  well  as,  the  other  cells  during 
the  one-year  period  of  the  test.  This  occurred 
even  though  the  other  three  cells  had  substantially 
higher  rates  of  advertising  expenditures.  Indeed, 
Cell  Red  had  a  much  higher  rate  of  Joint  spending 
while  Cells  Yellow  and  Green  had  higher  rates  of 
expenditure  for  Service-specific  advertising. 

This  chapter  provides  additional  analyses  of 
the  experimental  data.  It  is  aimed  at  answering 
two  questions: 

1 .  Why  did  Cell  Blue  perform  better  than,  or 
at  least  as  well  as,  the  other  cells  during 
the  one-year  period  of  the  test? 

2.  Do  insights  into  the  reasons  for  Cell 
Blue's  performance  provide  guidance  for 
recommendations  about  DoD  advertising 
budgets  and  policies? 

After  describing  the  findings,  this  chapter 
introduces  and  tests  three  hypotheses  that  may 
explain  Cell  Blue's  performance.  The  steps  taken 
to  validate  a  model  supporting  one  of  the  three 
hypotheses  is  then  described.  Additional  tests  are 
also  provided  to  confirm  one  of  the  hypotheses. 

SUMMARY  OF  RESULTS 

The  findings  in  this  chapter  can  be  sum¬ 
marized  as  follows: 

•  The  percentage  of  advertising  allocated  to 
Joint  was  not  the  only  factor  affecting  Cell 


Blue's  performance.  In  addition,  there 
was  no  evidence  that  the  recruiting  perfor¬ 
mance  in  Cell  Blue  was  a  result  of  only 
the  total  DoD  advertising  budget  level. 
The  enlistment  responses  to  the  adver¬ 
tising  treatment  observed  during  the 
experiment  were  a  function  of  both  the 
level  of  advertising  budgets  and  the  mix 
between  Service-specific  and  Joint 
advertising. 

The  total  level  of  advertising  expenditures 
and  the  mix  between  Service-specific  and 
Joint  expenditures  do  not  operate  indepen¬ 
dently  in  affecting  enlistment  behavior. 

Higher  proportions  of  Joint  advertising 
were  effective  in  obtaining  good  response 
at  lower  overall  expenditure  levels.  As  the 
budget  is  increased,  the  percentage 
allocated  to  Joint  to  maximize  response 
declines. 

Cell  Blue's  performance  during  the  test  is 
not  caused  by  an  underlying  relationship 
between  enlistment  response  and  adver¬ 
tising  that  is  perversely  negative.  Rather, 
cells  with  higher  budgets  and  less  efficient 
mixes  of  advertising  expenditures 
performed  worse  than  Cell  Blue  which 
had  a  lower  budget  and  better  mix  of 
advertising  expenditures. 

Insight  into  how  the  relationship  between 
the  level  of  advertising  expenditures  and 
the  mix  between  Service-specific  and  Joint 


m 


advertising  affects  the  experimental  results 
permits  budget  computations  and  recom¬ 
mendations  that  go  beyond  a  recommen¬ 
dation  to  implement  the  Cell  Blue 
treatment  condition  nationally. 
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ANALYSIS  OF  ADVERTISING 
RESPONSE 


'ii 


Three  hypotheses  were  investigated  regarding 
the  impact  of  total  advertising  expenditure  and  of 
the  proportion  of  that  expenditure  on  Joint 
advertising  on  recruitment  response.  The  aim  is 
to  test  for  the  independence  or  interdependence 
between  Joint  and  Service-specific  advertising. 


Total-Expenditure  Hypothesis 

The  first  hypothesis  for  explaining  the 
experimental  results  states  that  enlistment  behav¬ 
ior  responded  only  to  the  total  level  of  advertising 
expenditures  regardless  of  whether  the  expendi¬ 
tures  are  Joint  or  Service-specific.  This  implies 
that  the  target  audiences  do  not  differentiate  Joint 
from  Service-specific  advertising.  Counter  to  this 
hypothesis,  the  cell-based  experimental  results 
imply  that  an  inverse  relationship  exists  between 
advertising  expenditures  and  recruiting  perfor¬ 
mance  across  the  range  of  budgets  tested.  This  is 
schematically  represented  in  Figure  7.1.  Higher 
levels  of  advertising  lead  to  or  cause  lower  levels 


The  hypotheses  are: 

1.  Response  depends  only  on  total  adver¬ 
tising  expenditure 

2.  Response  depends  only  on  the  percentage 
allocated  to  Joint  advertising 

3.  Response  is  based  on  the  interaction 
between  the  total  advertising  expenditure 
and  the  percentage  allocated  to  Joint 
advertising 

These  hypotheses  were  investigated  using 
applicant  data  that  are  available  on  an  ADI  basis, 
thereby  increasing  the  number  of  observations. 
Applicant  data  are  less  affected  by  goals  and 
missions  than  are  data  on  contracts.  Ideally,  these 
hypotheses  should  also  be  tested  using  reported 
and  attitudinal  measures.  This  could  not  be  done 
because  reliable  measures  (with  sufficient  sample 
size)  of  these  data  were  available  only  at  the  cell 
level.  However,  the  general  consistency  of  the 
previously  reported  results  using  attitudinal  and 
reported  data  and  applicant  data  makes  such  an 
analysis  less  necessary. 
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of  enlistment  behavior. 

This  hypothesis  suggests  that  not  only  are 
increased  advertising  expenditures  not  contri¬ 
buting  to  increased  sales  (having  reached  some 
saturation  point)  but  also  high  levels  of  adver¬ 
tising  expenditures  actually  decrease  sales  (having 
reached  a  supersaturation  point).  The  implica¬ 
tions  of  this  hypothesis  in  the  current  experiment 
are  that  the  market  is  indifferent  to  the  mix 
between  Service-specific  and  Joint  advertising 
and  that  no  budget  level  above  die  total  Cell  Blue 
level  should  be  implemented.  Before  this 
hypothesis  can  be  accepted,  a  significant  negative 
relationship  between  total  advertising  expendi¬ 
tures  and  enlistment  behavior  should  be 
established.  A  number  of  different  models, 
described  in  Appendix  G,  were  formulated  and 
tested  on  the  ADI  data  to  assess  this  hypothesis. 
A  significant  negative  relationship  between  per 
capita  advertising  expenditures  and  per  capita 
applicant  rates  was  not  established  and  hence  the 
hypothesis  was  rejected.  The  conclusion  is  that 
the  enlistment  response  during  the  experiment  is 
not  solely  a  function  of  total  advertising 
expenditure  levels. 


Figure  7.1 


Enlistment  Response  to  Total  Advertising  Budget  Levels  in  the  Test  Cells 
(Total  Expenditure  Hypothesis) 


A  -  Cell  Yellow 


B«  Cell  Blue 


C  ■  Cell  Gxeen 


D  -  Cell  Red 


Percent-Allocated-to-Joint  Hypothesis 


The  second  hypothesis  states  that  enlistment 
behavior  responds  only  to  the  mix  of  advertising 
budgets  or  expenditures  between  Service-specific 
and  Joint,  irrespective  of  total  expenditure  or 
budget  level.  This  implies  that  Service-specific 
and  Joint  advertising  are  differentiated  by  the 
target  audiences  and  that  different  response  func¬ 
tions  exist  for  each  that  either  are  constant  at  all 
levels  of  spending  or  vary  with  expenditure  levels 
at  precisely  proportional  rates.  Given  this  hypo¬ 
thesis,  the  cell-based  experimental  results  imply 
that  a  non-monotonic  relationship  exists  between 
the  mix  of  advertising  budget  levels  and  recruiting 
performance  across  the  range  of  budgets  tested. 
This  is  represented  schematically  in  Figure  7.2. 
The  enlistment  response  to  advertising  rises  with 
the  percent  Joint  up  to  some  optimal  point, 
beyond  which  response  is  unchanged  or  declines. 


The  implication  of  this  hypothesis  in  the 
current  experiment  is  that  the  market  is  sensitive 
only  to  the  mix  between  Service-specific  and  Joint 
advertising.  In  addition,  the  mix  should  be  set  at 
or  near  the  Cell  Blue  rate  of  approximately  41 
percent  Joint  (based  on  per  capita  delivered 
advertising  expenditures).  Accepting  this  hypo¬ 
thesis  would  require  that  the  optimal  mix  between 
the  two  response  functions  be  independent  of  the 
level  of  advertising  expenditures  or  budget 
Again,  a  number  of  different  models,  described  in 
Appendix  G,  were  formulated  and  tested  on  the 
ADI  data  to  assess  this  hypothesis.  A  significant 
relationship  between  the  mix  of  advertising 
expenditures  observed  and  per  capita  applicant 
rates  was  not  established.  Hence,  the  hypothesis 
was  rejected.  The  conclusion  is  that  the  enlist¬ 
ment  response  during  the  experiment  is  not  solely 
a  function  of  the  mix  between  Service-specific 
and  Joint  expenditures  or  budget  levels. 


Figure  7.2 

Enlistment  Response  to  Percent  Joint  of  Total  Budget  Levels  in  the  Test  Cells 
(Percent  Allocated  to  Joint  Hypothesis) 


i 


A  -  Cell  Yellow 


20% 


B  -  Cell  Blue 


40% 


60% 


Percent  Joint 
in  the  Test  Ceils 


C  ■  Cell  Green 


80% 
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Interaction  Hypothesis 

The  third  hypothesis  states  that  the  total  level 
of  advertising  expenditures  and  the  mix  between 
Service-specific  and  Joint  expenditures  do  not 
operate  independently  in  affecting  enlistment 
behavior.  This  hypothesis  implies  that  Service- 
specific  and  Joint  advertising  are  differentiated  by 
the  target  audiences.  It  also  indicates  that  the 
appropriate  mix  between  the  two  areas  of 
advertising  will  depend,  at  least  in  part,  on  the 
overall  expenditure.  (Or,  alternatively,  that  the 
appropriate  expenditure  level  would  partially 
depend  on  the  desired  mix  between  the  two 
categories.)  Hence,  one  mix  between  Service - 
specific  and  Joint  advertising  may  be  appropriate 
and  effective  at  one  level  of  total  expenditures 
whereas  a  quite  different  mix  is  appropriate  at  a 
lower  or  higher  level  of  total  expenditures. 


To  understand  this  hypothesis,  it  is  helpful  to 
consider  the  following  numerical  example  Let 
total  response  depend  on  the  allocation  of  a  total 
budget  B  between  two  campaigns.  Campaign  1 
and  Campaign  2;  total  response  is  the  sum  of  the 
responses  obtained  from  each  campaign  The 
response  from  each  campaign  is  shown  in  Figure 
7.3.  Campaign  1  has  a  linear  response  to  per 
capita  spending  up  to  a  spending  rate  of  $3  The 
"saturation”  response  is  3  units.  Similarly. 
Campaign  2  has  a  saturation  response  of  2  units  at 
a  spending  rate  of  $S  per  capita.  These  response 
functions  are  simplified  to  ease  the  exposition  of 
response  functions  typically  found  in  marketing 
studies. 

Based  on  our  assumptions,  if  B  were  set  at  $8 
per  capita  and  S3  were  allocated  to  Campaign  I 
and  $5  went  to  Campaign  2,  the  total  response 
would  be  5  units.  Let  "p"  be  the  proportion  of  the 
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Figure  7J 


Two  Enlistment  Response  Functions  for  Campaigns  1  and  2 
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budget  B  aUocaaed  »  Campaign  1  Thus  when 
p  »  3/8.  local  response  is  S  The  local  response 
can  be  easily  calculated  for  different  allocations 
with  corresponding  values  of  "p"  and  for  different 
values  of  B  Table  7  l  shows  the  calculations  for 
B  *  H. 

The  relationship  between  "p"  and  total  re¬ 
sponse  is  graphed  in  Figure  7  4.  This  figure  also 
shows  similar  graphs  developed  for  values  of  B 
equal  lo  S4  and  $6  per  capita.  These  graphs  por¬ 
tray  an  imeracTuui  between  the  total  budget  B  and 
the  proportion  allocated  tc  Campaign  1.  As  B 
increases,  the  allocation  ”p~  required  for  Cam¬ 
paign  1  to  achieve  maximum  total  response 
decreases.  For  B  ■  4.  "p"  is  0.75,  while  for  B  * 
8.  it  is  0.375  This  result  ("p"  changing  with  the 
budget  level )  illustrates  the  interaction  hypothesis. 
Under  Hypothesis  1.  the  graphs  at  each  level  of  B 
would  be  parallel  to  the  horizontal  axis  producing 
the  same  response  for  a  given  B.  regardless  of 
”p"  (Figure  7.5).  For  Hypothesis  2.  there  would 
be  only  a  single  graph,  regardless  of  the  value  of 
B  (Figure  7.6). 


As  indicated  in  Figure  7.4,  total  response 
depends  on  both  the  level  of  total  advertising 
expenditures  and  on  the  proportion  allocated  to 
each  campaign.  Observe  the  two  points  labelled 
X  and  Y  in  Figure  7.4.  Note  that  point  Y 
represents  a  higher  response  despite  the  facts  that 
it  represents  a  lower  total  level  of  expenditures 
and  that  neither  Campaign  1  nor  Campaign  2 
exhibits  a  negative  relationship  between  response 
and  advertising  (see  Figure  7.3).  Point  X 
achieves  better  response  because  it  represents  a 
better  mix  between  Campaign  1  and  Campaign  2 
for  its  budget  level. 

To  test  the  interaction  hypothesis,  the  most 
direct  approach  is  to  estimate  the  two  response 
functions  (one  for  Service-Specific  advertising 
and  one  for  Joint  advertising)  separately. 
Alternatively,  it  can  be  tested  indirectly  by 
estimating  curves  such  as  those  in  Figure  7.4  for 
different  budget  levels.  Because  of  the  highly 
constrained  nature  of  the  test  design,  neither 
approach  is  easy.  For  example,  only  two  levels 
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Table  7.1 


Total  Enlistment  Response  as  a  Function  of  p 
(p  =  percent  of  total  budget  expenditure  to  Campaign  1) 


Allocation  to  Campaign  1 

0 

1 

2 

3 

4 

5 

6 

7 

8 

Allocation  to  Campaign  2 

8 

7 

6 

5 

4 

3 

2 

1 

0 

Total  Budget  (B) 

8 

8 

8 

8 

8 

8 

8 

8 

8 

p  (percent  of  Campaign  1 
expenditure  to  Total  Budget) 

0 

.125 

.25 

.375 

.5 

.625 

.75 

.875 

1 

Response  1 

0 

1 

2 

3 

3 

3 

3 

3 

3 

Response  2 

2 

2 

2 

2 

1.6 

1.2 

.8 

.4 

0 

Total  Response 

2 

3 

4 

5 

4.6 

4.2 

3.8 

3.4 

3 

Figure  7.4 

Interaction  Hypothesis:  Enlistment  Response  Functions 
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Figure  7.5 


Hypothesis  I:  Enlistment  Response  Depends  Only  on  Total 
Advertising  Expenditure  (B) 


Hypothesis  H:  Enlistment  Response  Depends  Only  on 
Percent  Allocated  to  Joint  (p) 
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of  spendirj  finr  Service-specific  advertising  were 
tested  ($68  million  and  $15  million),  making  the 
estimation  of  a  response  function  difficult  Also, 
as  Figure  7.7  shows,  the  proportion  of  Joint 
advertising  to  the  total  budget  that  is  observed  is 
not  as  dense  and  uniform  as  would  be  ideal.  For 
example,  there  are  few  points  with  expenditure 
rates  of  $5  or  more  and  a  percentage  of  Joint 
advertising  in  the  40-60  percent  range.  Similarly, 
there  are  very  few  observations  with  the 
percentage  of  Joint  advertising  greater  than  80 
percent  or  less  than  20  percent  Thus,  estimation 
of  the  complete  contour  of  the  response  curves,  as 
in  Figure  7.4,  is  impossible. 


Based  on  preliminary  data  analysis,  the 
interaction  hypothesis  was  tested  indirectly  by 
estimating  portions  of  the  response  curves.  This 
testing  is  constrained  by  the  data  as  noted  above. 

The  data  were  first  divided  according  to  the 
level  of  advertising  expenditure:  low  (less  than 
$5.50  per  17-21-year-old  male  for  year  of  the 
test),  medium  (between  $5.50  and  $7.00),  and 
high  (greater  than  $7.00).  Each  ADI  in  the 
experiment  was  classified  into  one  of  these  total 
expenditure  categories.  This  resulted  in  the 
creation  of  three  separate  data  sets  of  roughly 
equal  size  corresponding  to  three  separate  levels 
of  total  advertising  expenditures. 


Figure  7.7 


ADI  Advertising  Expenditure  Level  by  the  Proportion 
Allocated  to  Joint 
(Oct/1983-Sep/1984) 
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These  three  data  sets  provided  the  structure 
for  assessing  the  interaction  hypothesis.  With  the 
level  of  total,  advertising  expenditures  now  fixed, 
attention  could  be  turned  to  the  effect  of  the  mix 
between  Joint  and  Service-specific  expenditures 
on  enlistment  response  for  each  total  expenditure 
category.  The  essence  of  the  interaction  hypo¬ 
thesis  is  that  the  enlistment  response  to  the  mix 
between  Joint  and  Service-specific  advertising 
varies  at  different  total  advertising  expenditure 
levels.  This  premise  could  now  be  assessed. 
Models  relating  DoD  applicants  and  Army  appli¬ 
cants  (total  as  well  as  high  school  diploma 
graduates  or  seniors)  to  the  percent  Joint  were 
developed  for  each  of  the  three  expenditure 
categories.  The  effect  of  the  percentage  of  Joint 
advertising  to  total  advertising  on  applicants  was 
then  compared. 

Table  7.2  presents  the  results  of  the  models  of 
applicants.  The  dependent  variable  (Y)  was  appli¬ 
cants  per  1,000  17-21 -year-old  males  in  an  ADI. 
The  independent  variables  were  P  (the  proportion 
of  Joint  advertising  expenditures  to  the  total 
advertising  expenditures  in  a  market)  and  R  (the 
number  of  recruiter  person-months  of  effort  in  a 
market).  All  observations  were  on  an  annual 
basis. 

Figure  7.4  shows  that  the  proportion  of  a 
given  level  of  total  advertising  expenditure  allo¬ 
cated  to  a  given  component,  such  as  Joint 
advertising,  can  result  in  an  increasing  response 
up  to  a  certain  "turning"  point.  After  that  point, 
diminishing  responses  are  obtained.  The  formu¬ 
lation  of  models  designed  to  reflect  this 
characteristic  explicitly  was  deterred  by  two 
considerations.  First,  the  data  were  limited  by  a 
lack  of  observations  (especially  where  Joint  adver¬ 
tising  was  greater  than  80  percent  of  a  low  total 
budget  and  where  Joint  advertising  was  less  than 
20  percent  of  a  high  total  budget).  In  addition. 


relatively  few  total  observations,  and  hence 
degrees  of  freedom,  were  available  at  any  of  the 
total  expenditure  levels.  It  is  possible,  therefore, 
that  the  "turning  point"  lies  within  or  close  to  the 
range  of  unobserved  data  and  would  not  be 
properly  estimated.  Second,  while  desirable,  it  is 
not  necessary  to  estimate  precisely  the  response 
curves  of  applicants  to  the  various  advertising 
mixes  for  each  level  of  total  advertising 
expenditure.  The  interaction  hypothesis  can  be 
sufficiently  assessed  by  estimating  the  major 
segments  of  the  response  functions  for  which  data 
are  available. 

Before  the  interaction  hypothesis  can  be 
accepted  (or  rejected),  it  is  necessary  to  establish 
(or  reject)  that  enlistment  response  to  the  mix 
between  Joint  and  Service-specific  advertising  is 
significantly  different  at  the  various  levels  of  total 
advertising  expenditure.  Hence,  monotonic 
models  of  enlistment  response  were  estimated.  In 
such  models,  the  relationship  between  enlistment 
response  and  the  percentage  of  Joint  advertising  is 
strictly  increasing  or  strictly  decreasing  for  the 
segment  of  the  response  curve  being  estimated. 

In  these  models,  a  significantly  positive 
coefficient  estimate  for  the  P  (percentage  Joint) 
term  indicates  that  a  positive  relationship  exists 
between  the  percentage  of  Joint  advertising  and 
the  applicant  response  rate  across  the  observations 
of  the  total  level  of  advertising  expenditures. 
Enlistment  response  increases  as  the  percentage  of 
Joint  advertising  increases  for  the  observations 
modeled.  Conversely,  a  significantly  negative 
coefficient  estimate  for  the  P  term  indicates  a 
negative  relationship  between  the  percentage  of 
Joint  advertising  and  the  applicant  response  rate 
for  the  segment  modeled.  Because  only  major 
segments  of  (not  the  entire)  response  functions 
are  modeled,  enlistment  response  to  the  percen¬ 
tage  of  Joint  advertising  may  change  direction 


Table  7.2 


Annual  Cross  Sectional  Models  of  DoD  Applicant  Rates 
At  Different  Total  Budget  Levels 


Functional  Form 

Coefficient  Estimates 

Measure  of  Fit 

Y  *  a  +  b  log  (P)  +  cR 

a  b 

c 

R2 

Low  Spending  T  £  5.50 

1.63  .37* 

2.30** 

.40 

High  Spending  T  2  7.01 

.87  -.51** 

1.81** 

.58 

Y  =*  a  +  bP2  +  cP  +dR 

a  b  c 

d 

R2 

Medium  Spending 

5.50  <  T  <  7.01 

2.26  -3.03  2.67 

.57** 

.28 

T  =  Total  working  media  expenditures 

Y  »  Total  DoD  applicants  per  1000  17-21-year-old  males 

P  =  Proportion  Joint  advertising  of  total  budget 

R  =  Recruiter  person-months 

*  -  Statistic  significant  at  0.10  level. 
**  -  Statistic  significant  at  0.05  level. 


toward  the  end  of  or  beyond  the  range  of  data 
observed  (even  given  a  statistically  significant 
coefficient  for  the  P  term). 

The  results  of  the  models  of  applicant 
response  estimated  for  the  high  and  low  total 
advertising  expenditure  levels  support  the 
interaction  hypothesis.  As  displayed  in  Table 
7.2,  a  positive  coefficient,  significantly  different 
from  zero  at  the  0.10  level,  is  estimated  for  the 
percentage  of  Joint  advertising  at  the  low  total 
advertising  expenditure  level  for  the  segment  of 
observations  available.  Similarly,  a  negative 
coefficient,  significantly  different  from  zero  at  the 


0.05  level,  is  estimated  for  the  percentage  of  Joint 
advertising  at  the  high  total  advertising  expendi¬ 
ture  level.  Clearly,  these  coefficient  estimates 
differ  from  one  another. 

The  significant  positive  relationship  of  appli¬ 
cant  response  to  the  percentage  of  Joint 
advertising  at  the  low  total  advertising  budget 
implies  that  the  greatest  response  is  obtained  near 
the  end  of  the  observed  range  (about  80  percent 
Joint  advertising).  Similarly,  the  significant  nega¬ 
tive  relationship  of  applicant  response  to  the 
percentage  of  Joint  advertising  at  the  high  total 
budget  implies  that  the  greatest  response  is 
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obtained  near  the  beginning  of  the  observed  range 
at  this  level  of  advertising  (about  20  percent  Joint 
advertising).  Extrapolating  these  models  beyond 
the  range  of  the  observed  data  is  highly  inappro¬ 
priate.  In  particular,  one  should  not  conclude, 
based  on  these  models,  that  the  maximizing 
percentage  of  Joint  advertising  is  zero  at  the  high 
total  expenditure  level  nor  one  hundred  at  the  low 
level. 

For  the  middle  level  of  total  advertising 
expenditures,  an  additional  modeling  step  was 
undertaken.  Few  observations  of  the  middle 
range  of  Joint  advertising  were  available  for  this 
level  of  total  advertising  expenditures.  Hence,  a 
non-monotonic  model  was  formulated  to  allow 
for  a  turning  point  to  occur  in  the  largely 
unobserved  area.  Coefficients  estimated  for  the 
percentage  of  Joint  advertising  were  not  signifi¬ 
cantly  different  from  zero  for  either  the  monotonic 
or  non-monotonic  model.  The  non-monotonic 
model  did,  however,  provide  an  estimate  of  the 
turning  point 

Key  advantages  of  the  basic  modeling 
approach  described  here  are  that  it  permits  the  pre¬ 
dictive  validation  steps  reported  in  the  following 
section.  It  also  provides  guidance  for  budget 
computations  reviewed  later  in  this  chapter. 

In  summary,  the  hypothesis  that  the  total  level 
of  advertising  expenditures  and  the  mix  between 
Service-specific  and  Joint  expenditures  do  not 
operate  independently  in  affecting  enlistment 
behavior  has  been  tested.  The  effect,  on  the  rate 
of  applicants,  of  the  percentage  of  total  adver¬ 
tising  expenditures  allocated  to  Joint  advertising 
was  found  to  be: 

•  Positive  and  significantly  different  from 
zero  at  the  low  level  of  total  expenditures 

•  Not  significantly  different  from  zero  at  the 
medium  level  of  total  expenditures 


•  Negative  and  significantly  different  from 
zero  at  the  high  level  of  total  expenditures 

The  conclusion  is  that  the  total  level  of 
advertising  expenditures  and  the  mix  between 
Joint  and  Service-specific  expenditures  do  interact 
and  do  not  operate  independently  in  affecting 
enlistment  behavior.  Higher  proportions  of  Joint 
advertising  are  effective  at  lower  levels  of  total 
expenditures.  Lower  proportions  of  Joint 
advertising  are  appropriate  at  higher  levels  of  total 
expenditures. 

PREDICTIVE  VALIDATION 

The  fitted  models  provide  support  for  the 
hypothesis  of  interaction  between  the  size  of  the 
advertising  budget  and  its  allocation  between  Joint 
and  Service- specific  programs.  Much  stronger 
support  can  be  provided  by  a  predictive  test 
Although  any  number  of  model  formulations 
might  fit  a  given  set  of  data,  for  policy-making 
purposes,  assurance  is  needed  that  any  such 
model  would  also  perform  well  on  an  independent 
data  set  Such  an  independent  data  set  could  be 
gathered  simultaneously  with  the  given  set  and 
"held  out"  from  the  fitting  procedure,  or  it  might 
be  obtained  from  a  different  time  period.  In  this 
case,  a  holdout  sample  was  not  feasible  because 
of  limited  numbers  of  ADIs  available.  Therefore, 
the  second  approach  was  adopted. 

Ideally,  1983  data  could  be  used  to  fit  the 
models.  Then  using  actual  1984  spending  and 
recruiting  resources  in  each  ADI,  the  response 
could  be  forecast  (e.g.,  total  applicants.  Army 
applicants)  by  ADI.  In  addition,  the  mean 
response  by  cell  could  be  computed.  Finally,  it 
would  be  possible  to  determine  whether  differ¬ 
ences  between  the  means  of  one  cell  versus  the 
control  cell  matched  similar  differences  obtained 
using  actual  response  (all  other  factors  remaining 


comparable).  Using  models  developed  on  1984 
data,  responses  in  1983  could  be  predicted,  given 
actual  1983  spending  and  recruiting  resources. 

A  departure  from  this  idealized  procedure  in 
the  predictive  validation  test  was  necessary. 
Indeed,  very  substantial  differences  existed 
between  1983  and  1984  actual  responses  that 
were  unrelated  to  the  advertising  budget  or  its 
allocation.  It  was  assumed  that  these  differences 
in  response  were  a  result  of  economic  and  related 
factors  as  indicated  in  Chapter  2. 

To  compensate  for  these  changes,  the 
idealized  procedure  was  modified  as  follows: 

•  First,  1983  advertising  expenditures  were 
inflated  by  7  percent  in  order  to  make  a 

1983  advertising  dollar  equivalent  to  a 

1984  advertising  dollar.  All  further 
computations  were  conducted  with  these 
equivalent  dollars. 

•  The  ADIs  were  classified  into  three 
groups:  low,  medium,  and  high  adver¬ 
tising  expenditure  rates. 


the  Philadelphia,  PA  ADI  in  1984, 
basedon  the  fitted  model. 

•  Similarly,  let  Ay  denote  the  actual  value  of 
the  jth  response  in  ADI  i  in  1983,  and  let 
By  denote  the  corresponding  quantity  in 
1984.  For  example,  these  values  for  the 
Philadelphia,  PA  ADI  might  be  5  and  3.5 
in  1983  and  1984,  respectively. 

•  We  calculated  the  average  Fy,  Gy,  Ay, 
and  By  for  each  response  j  across  the 
ADIs  in  each  experimental  cell.  Let  the 
means  of  these  cells  be  denoted  by 
Gkj,  Aty  and  B)y,  where  k  *  Cell  Yellow, 
Blue,  Green  and  Red.  The  means  of  these 
cells  are  displayed  in  Table  7.3  for  DoD 
total,  DoD  high  school  and  senior  appli¬ 
cants,  Army  total  and  Army  high  school 
and  senior  applicants.  There  is  a  generally 
close  agreement  between  the  fitted  and 
actual  cell  means  and  a  wide  difference 
between  forecast  and  actual  1983  cell 
means.  These  differences  are  not  attri¬ 


butable  to  the  experimental  treatments. 

•  "Forecasts"  of  applicant  response  were 

made  for  each  ADI  for  1983  using  the  To  validate  the  model,  two  sets  of  changes 

equivalent  dollar  expenditures  on  adver-  from  1983  to  1984  were  calculated,  by  cell  and 
rising  and  actual  recruiter  resource  ailoca-  type  of  response  based  on: 

rions.  Let  these  be  designated  by  Fy 


where  i  is  the  ADI  number,  and  j  the  type 
of  response.  For  example,  the  predicted 
response  for  total  applicants  for  the 
Philadelphia,  PA  ADI  in  1983  might  be  6 
per  1,000  17  -  21-year-old  males. 

Let  Gy  denote  the  fitted  value  of  the  jth 
response  in  ADI  i,  using  the  1984  data. 
For  example,  this  might  be  4  total  appli¬ 
cants  per  1,000  17-21 -year-old  males  for 


•  the  model 

•  only  the  data 

The  changes  based  on  the  model  do  not  reflect  the 
impact  of  the  economic  variables  whereas  those 
based  on  the  actual  data  do.  Thus,  the  two  sets  of 
changes  are  not  yet  comparable.  The  difference  in 
response  between  each  cell  and  control  for  each 
set  was  computed.  The  assumption  (based  on  the 
market-matching  which  used  geographic  and 


Table  7 3 


Actual  Compared  to  Fitted  and  Forecast  Applicant  Rates 
for  Selected  Enlistment  Group  by  Test  Ceil 


1984 

1983 

skj 

Rkj 

1 

Actual  B^j 

Fitted  Gig 

Actual  A^j 

Forecast 

Bkj 
~  Akj 

t?|iF 

II 

m 

DoD  Total 

ft 

k 

Yellow 

3.95 

3.97 

4.93 

3.96 

.801 

1.000 

S 

St 

Blue 

4.06 

4.11 

431 

3.91 

.844 

1.051 

i 

ft 

Green 

3.81 

3.77 

4.76 

4.05 

.800 

0.931 

Red 

3.96 

3.94 

4.97 

3.96 

.797 

0.999 

DoD  HS/SR 

r. 

r, 

A 

k 

1.006 

a 

Yellow 

3.32 

3.34 

4.19 

3.32 

.792 

$ 

8? 

Blue 

3.38 

3.43 

4.06 

3.29 

.833 

1.043 

J 

Green 

3.22 

3.17 

4.01 

3.40 

.803 

0.932 

Red 

3.39 

336 

4.16 

3.32 

.815 

1.012 

Army  Total 

S 

k 

Yellow 

1.85 

US 

232 

1.82 

.833 

1.016 

«v 

Bine 

1.88 

1.92 

2.11 

1.74 

.891 

1.103 

*1 

* 

Green 

1.72 

1.68 

2.16 

1.82 

.796 

0.923 

r 

t 

A 

Red 

1.78 

1.78 

231 

1.78 

.805 

1.000 

r 

r 

i 

Army  HS/SR 

r 

< 

a 

k 

Yellow 

1.44 

1.43 

1.74 

1.40 

.828 

1.021 

r 

Blue 

1.46 

130 

1.64 

1.34 

.890 

1.119 

V 

■jJ3 

Green 

1.30 

137 

1.64 

1.40 

.793 

0.907 

s 

Red 

1.42 

1.42 

1.73 

1.38 

.821 

1.029 

% 

* 

% 

HS/SR  -  high  school  and  senior  applicants 


economic  data)  that  die  economic  effect  is  equal  in 
each  cell  was  then  made.  These  economic  effects 
were  removed  from  the  changes  based  on  actual 
data  by  calculating  these  differences.  Thus,  the 
cell  differences  obtained  using  changes  based  on 
the  model  and  those  based  on  actual  data  became 
comparable.  As  will  be  shown  below,  the  two 
sets  of  numbers  are  extremely  close  both  for  total 


DoD  responses  and  for  Army  responses,  thus 
providing  support  for  the  model. 

The  ratios  =  Gy  /  were  then  com¬ 
puted.  These  are  the  proportional  changes 
between  1983  and  1984  in  the  cells  k  =  Yellow, 
Blue,  Red  and  Green,  for  the  various  types  of 
response  j.  These  ratios  can  be  expressed  as 
percentages  by  multiplying  them  by  100. 


An  example  of  this  ratio  is 

^Yellow,  Total  Applicants  ~ 

Model  fit  for  1984  Cell  Yellow, 

Total  Applicants 

Model  forecast  for 
1983  Cell  Yellow,  Total  Applicants 

We  then  subtracted  RyeUow,  j  fr°ra  Rail*.  j» 
RRed,  j  and  Reran,  j  in  order  to  obtain  the  model 
based  cell  effects  for  response  j.  For 
convenience,  let 

Ckj  =  Rkj  ‘  ^Yellow  j  • 

Thus 

^Yellow,  Total  Applicants  =  and 

^Blue.  Total  Applicants  *  RbIuc,  Total  Applicants 
‘  ^Yellow,  Total  Applicants* 


These  quantities,  for  various  types  of 
responses,  are  displayed  in  Table  7.4. 

A  similar  set  of  numbers  was  computed, 
using  Sy  =  By  /  Ay.  The  formula  represents  the 
proportional  changes  between  1983  and  1984  in 
the  cells  k  =  Yellow,  Blue,  Green  and  Red  for  the 
various  types  of  response  j.  For  example, 

S  Yellow,  Total  Applicants 

represents  the  ratio  of  actual  response  in  1984 
Cell  Yellow  by  total  applicants  to  actual  response 
in  1983  Cell  Yellow  by  total  applicants. 

The  quantities  Dy  =  Sy  -  Syell0Wi  j  arc 
defined  to  obtain  the  actual  values  of  the  cell 
effects.  These  quantities  are  also  displayed  in 
Table  7.4. 


Table  7.4 

Percentage  Difference  from  the  Control  Cell  in  Applicant  Rates 

across  the  Test  Cells 


Total  Applicants 

HS/SR  Applicants 

Predicted  Ckj 

Actual  Dfcj 

Predicted  Ckj 

Actual  D^j 

k 

Yellow 

0% 

0% 

0% 

0% 

Blue 

5.1% 

4.3% 

3.7%* 

4.0% 

Green 

-7.0% 

-.1% 

-7.4% 

1.1% 

Red 

-.i% 

.4% 

.6% 

2.3% 

ARMY 

Total  Applicants 

HS/SR  Applicants 

Predicted  C^j 

Actual  Dkj 

Predicted  C^j 

Actual  Djg 

k 

Yellow 

0% 

0% 

0% 

0% 

Blue 

8.7%** 

6.8% 

9.8%** 

6.2% 

Green 

-9.3% 

-3.7% 

-11.4% 

-3.5% 

Red 

-1.6% 

-2.8% 

.8% 

-.7% 

HS/SR  -  high  school  and  senior  applicants 

*  Significantly  different  from  control  at  the  0.10  level. 
**  Significantly  different  from  control  at  the  0.05  level. 


ADDITIONAL  TESTING  OF  THE 
INTERACTION  HYPOTHESIS 


In  order  to  provide  additional  support  for  the 
interaction  hypothesis,  and  to  obtain  results  that 
are  as  independent  as  possible  of  the  specific 
formulation  used,  several  other  analyses  of  the 
experimental  data  were  conducted.  In  a  manner 
analogous  to  the  analysis  of  the  observed 
behavior  measure,  experimental  markets  were 
regrouped  into  cells  defined  by  the  two  dimen¬ 
sions  of  interest  (the  recommended  levels  of  Joint 
versus  Service  advertising).  In  addition,  dummy 
variable  regression  models  were  fit  to  this  data. 
Detailed  results  of  several  of  these  models  are 
presented  in  Appendix  H.  These  results  also 
support  the  interaction  hypothesis. 


IMPLICATIONS  OF  ADDITIONAL 
ANALYSIS 


Alternative  implications  to  implementing  the 
Cell  Blue  advertising  expenditure  level  and  mix 
on  a  national  basis  are  warranted  to  the  extent  that 
the  data  and  analytical  tools  available  to  derive 
them  are  sufficient  As  discussed  above,  there  are 
indeed  limits  to  the  density  and  uniformity  of  data 
available  for  the  analyses  discussed  in  this 
chapter.  Consideration  of  policy  and  budget 
implications  based  on  these  analyses  is  appro¬ 
priate  because  incremental  system  effectiveness  is 
indicated  by  the  results.  In  other  words,  policy 
actions  based  on  the  analyses  discussed  in  this 
chapter  indicate  a  level  of  system  performance 
(measured  by  applicant  rates)  that  is  superior  to 
either  the  current  or  direct  implication  policy. 
Given  the  limitations,  however,  the  budget  impli¬ 
cations  are  expressed  as  ranges.  The  analyses 
indicate  that  several  different  mixes  and  levels  of 
advertising  expenditure  can  contribute  relatively 
equally  to  fulfilling  recruiting  system  require¬ 


ments  over  a  one-year  period.  Importantly,  these 
different  levels  and  mixes  would  entail  different 
levels  of  advertising  costs.  Again,  an  adaptive 
approach  to  implementation  is  appropriate. 

Specifically,  the  policy  implications  based  on 
these  analyses  are: 


Reduce  advertising  working  media 
spending  to  between  $4.35  and  $5.50  per 
17-21 -year-old  male  in  1984  dollars  from 
the  FY  1984  control  level  of  $7.00. 


Change  the  mix  of  advertising 
expenditures  (working  media)  between 
Joint  and  Service-specific  advertising  to 
between  45%  and  70%  Joint  from  the  FY 
1984  control  level  mix  of  17%. 


Reduce  the  FY  1987  DoD  enlisted,  active 
force  advertising  budget  from  the 
proposed  $154  million  to  between  $92 
million  and  $1 12  million.  Reduce  the  FY 
1987  total  DoD  advertising  budget  from 
the  proposed  $242  million  to  between 
$182  million  and  $201  million. 


Increase  the  FY  1987  Joint  advertising 
enlisted,  active  force  (working  media) 
budget  from  the  proposed  $23.1  million  to 
between  $35  million  and  $42  million. 


Provide  some  markets  (ADIs)  at  adver¬ 
tising  levels  that  are  both  significantly 
higher  and  significantly  lower  than  the 
levels  implied  by  the  above  budgets. 


Continue  research  and  analyses  to  under¬ 
stand  better  the  relationship  between  speci¬ 
fic  measures  of  enlistment  intentions  and 
subsequent  actual  enlistment  behavior. 


,** ,**  •**  *%i 


These  budgeting  guidelines  were  obtained 
from  the  models  fitted  and  described  in  this 
chapter.  Because  the  models  fit  to  the  low- 
spending  group  and  high  spending  group  are 
monotonically  increasing  and  decreasing  with  "p" 
(percentage  Joint  advertising),  respectively,  these 
models  would  suggest  that  "p"  should  be  close  to 
the  observed  highest  and  lowest  values  in  the 
respective  groups  of  markets.  From  the  middle 
spending  group  (overall  spending  rate  of  between 
$5.50  and  $7.00  per  17-21 -year-old  male  with  an 
average  of  $6.25),  to  maximize  total  DoD  appli¬ 
cants,  "p"  should  be  set  at  41  percent  To 
maximize  DoD  high  school  diploma  graduate  and 
senior  (HSDG  +  HS),  total  Army  and  Army 
HSDG  plus  HS,  "p"  should  be  set  at  42  percent, 
41  percent,  and  44  percent,  respectively. 
Selecting  42  percent  as  the  value  of  "p"  and 
applying  it  to  the  average  spending  rate  for  this 
group  of  markets  yields  a  suggested  spending  rate 
for  Joint  advertising  of  about  $2.60  per  capita. 
For  the  low-spending  ADIs  with  a  mean  spending 
rate  of  $4.34,  this  level  of  Joint  spending 
translates  to  a  value  of  Hp"  of  about  60  percent. 
Response  in  these  markets  increases  with  "p,"  but 
the  highest  observed  value  of  "p"  is  about  70 
percent  Any  possible  decline  in  response  toward 
the  end  of  the  observed  range  of  "p"  is  unlikely  to 
be  detected  by  our  model.  Thus,  a  value  of  60 
percent  for  "p"  seems  plausible. 

For  the  high-spending  ADIs  with  a  mean 
spending  rate  of  $7.98  per  capita,  the  middle¬ 
spending  group  translates  to  a  value  of  30  percent 
for  "p."  Response  in  these  markets  decreases 
with  "p,"  but  the  smallest  value  of  "p"  observed  is 
about  20  percent.  Thus,  a  30-percent  value  seems 
plausible. 

To  summarize,  point  estimates  of  suggested 
"p”  values  were  estimated  for  each  of  the  three 
working  media  spending  levels  as  follows: 


High-Expenditure  Level  30  Percent 

Joint 

Medium-Expenditure  Level  42  Percent 

Joint 

Low-Expenditure  Level  60  Percent 

Joint 

Since  no  significant  positive  advertising  effects 
were  supported  in  any  analysis  for  working  media 
expenditures  over  $5.50  per  17-21 -year-old  male, 
the  low-expenditure  level  is  recommended.  In 
other  words,  the  low-expenditure  level  of 
advertising  is  indicated  because,  over  the  period 
of  the  experiment,  no  additional  response  to 
advertising  in  excess  of  this  expenditure  level  was 
evidenced.  The  level  of  working  media  per  17-21- 
year-old  male  was  established  at  between  $4.34 
(the  mean  of  all  low-expenditure  level  obser¬ 
vations)  and  $5.50  (the  maximum  of  the  low- 
expenditure  category).  The  mix  of  working 
media  was  established  at  between  45  percent  (near 
the  suggested  point  estimate  for  the  medium 
expenditure  category)  and  70  percent  (the 
maximum  of  the  low-expenditure  observations). 

Moving  from  per  capita  FY  1984  working 
media  implications  to  actual  FY  1987  dollar 
budget  guidelines  required  several  steps.  Non¬ 
media  costs,  price  inflation  in  advertising  costs 
between  1984  and  1987,  and  non-test  advertising 
elements  had  to  be  computed  and  added  to  a 
working  media  aggregate  budget  determined  by 
multiplying  the  per  capita  spending  amounts  by 
9,677,000  (the  number  of  17-21-year-old  males 
in  1984).  Non-media  costs  were  interpolated 
from  data  provided  in  the  advertising  plans 
prepared  by  each  advertiser  in  preparation  for  the 
experiment.  These  costs  ranged  from  less  than  8 
percent  of  working  media  expenditures  for  Joint 


percent  of  working  media  expenditures  for  Joint 
advertising  at  its  highest  budget  level  to  more  than 
39  percent  of  working  media  expenditures  for 
Service-specific  advertising  at  its  lowest  level  of 
the  recommended  range.  These  costs  were  11.2 
percent  of  working  media  for  Joint  advertising  at 
the  control  budget  level  in  FY  1984  compared 
with  25.3  percent  for  the  Services  at  the  control 
budget  level.  These  additional  advertising 
program  costs  incurred  by  the  Services  were 


then  included  in  the  budget  computations. 
Advertising  price  inflation  was  estimated  at  10 
percent  per  year,  and  non-test  advertising  costs 
were  added  directly  back  into  budget  totals  where 
indicated  (i.e.,  reserve  and  officer  program 
advertising  were  added  back  into  the  recom¬ 
mended  total  DoD  advertising  budget  levels). 
Table  8.1  in  the  next  chapter  presents  the  results 
of  these  calculations  for  selected  budget  combi¬ 
nations  within  the  range  of  combinations  implied 
by  the  analyses  discussed  in  this  chapter. 


CHAPTER  8 


SUMMARY  AND  RECOMMENDATIONS 


Key  aspects  of  the  DoD  Advertising  Mix  Test, 
its  results  and  budget  implications  are  described  to 
consolidate  the  material  of  earlier  chapters  and  to 
support  the  recommendations  presented  here. 

The  DoD  Advertising  Mix  Test  was  an  in¬ 
market  test  designed  to  generate  quantitative  data 
to  help  answer  the  following  question:  "What  is 
the  optimum  mix  of  Joint/Service-specific  recruit¬ 
ment  advertising  for  achieving  active,  enlisted, 
non-prior  Service  (NPS)  goals  at  different  levels 
of  total  DoD  recruitment  advertising?"3  This 
question  was  addressed  with  a  field  experiment  to 
assure  meaningful  variance  in  the  levels  and 
mixes  of  advertising  observed,  statistical  indepen¬ 
dence  between  advertising  and  enlistments  in  past 
periods  (and  other  recruiting  variables),  and 
consistent  measurement  of  recruiting  system 
performance.  These  assurances  are  necessary  to 
allow  causal  inferences  to  be  drawn.  The  test  was 
collaboratively  designed  and  fielded  to  measure 
the  effect  of: 


Different  levels  of  total  DoD  advertising 
budgets 


Different  mixes  of  Service-specific  and 
Joint  advertising  budgets 


Hence,  the  test  focused  on  aggregate  budget 
levels.  All  allocation,  media,  and  placement 
decisions  were  made  and  implemented  by  the 
military  advertisers  through  the  same  decision  and 
control  processes  now  prevalent  in  military 
advertising  practice. 

The  test  involved  four  systematically  different 
budgets.  Each  was  implemented  on  a  pro-rated 


basis  in  a  test  cell  made  up  of  television  markets 
(ADIs)  and  identified  by  a  color.  The  four  test 
cells  were  matched  as  closely  as  possible  to  one 
another  for  past  enlistment  performance,  levels  of 
unemployment,  youth  propensity  to  enlist,  and 
for  racial,  urban  and  geographic  composition. 
The  test  budgets  were  implemented  in  the  test 
cells  for  a  period  of  one  year.  The  experimental 
budget  conditions  were  generally  well  imple¬ 
mented  by  the  military  advertisers  and  a 
comprehensive  data  set  covering  the  experimental 
period  was  assembled  and  analyzed. 

The  analysis  focused  on  providing  manageri- 
ally  useful  guidelines  for  DoD  advertising  budget 
policy  based  on  assessments  of  efficiency  and 
effectiveness  criteria.  Multiple  measures  of 
recruiting  system  performance  were  reviewed, 
collected  and  employed  in  the  analysis. 
Consistency  of  results  across  measures  and 
analysis  methods  was  emphasized.  The  direction 
and  order  of  magnitude  of  effective  changes  in 
budget  policies  were  sought  rather  than  precise 
and  static  decision  rules. 


RESULTS 


As  noted  in  Chapter  6,  Cell  Blue  performed 
better  than  or  at  least  as  well  as  the  other  cells. 
Cell  Blue,  with  the  lowest  overall  advertising 
expenditure,  did  not  adversely  affect  recruitment 
performance  during  the  test  year.  In  fact.  Cell 
Blue  performed  significantly  better  than  the  other 
cells  when  measured  by  enlistment  contract  and 
applicant  rates  per  unit  of  population. 


3Korb,  op.  cit. 


The  fact  that  Cell  Blue,  with  the  smallest 
advertising  budget,  did  better  than  the  other  cells, 
does  not  imply  that  advertising  is  ineffective  for 
military  recruiting.  Indeed,  changing  shares  of 
recruiting  advertising  allocated  to  the  various 
Services  could  yield  different  responses.  Recall 
that  the  shares  of  Service-specific  advertising 
budgets  were  intentionally  kept  stable  during  the 
test  period.  Furthermore,  only  the  level  and  mix 
of  expenditures  were  varied.  Advertising  was  not 
eliminated.  Consequently,  the  overall  effective¬ 
ness  of  advertising  for  military  recruiting,  beyond 
the  levels  and  mixes  tested,  cannot  be  assessed. 

The  results  of  the  experiment  are  sound  for 
the  military  recruiting  system  as  a  whole  and  are 
also  valid  for  the  Army  as  well  as  for  the  other 
Services. 

BUDGET  IMPLICATIONS 

The  results  of  the  experiment  have  budget 
implications  in  two  ways.  Based  only  on  the 
experimental  results,  a  national  implementation  of 
the  Cell  Blue  budget  levels  is  warranted  on 
grounds  of  both  efficiency  and  effectiveness. 
Any  such  implementation  should  be  accompanied 
by  a  commitment  to  the  notion  that  market 
response  to  the  new  budget  levels  should  be 
tracked  and  tested  further.  Some  markets  should 
be  maintained  at  alternative  spending  levels  to 
determine  the  longer-term  effects  of  the  Cell  Blue 
budget  level  This  alternative  is  referred  to  as  the 
direct  implications  of  the  experiment  Its 
implementation  would  lead  to  a  reduction  in  the 
FY  1987  DoD  enlisted,  active-force  advertising 
budget  from  $154.3  million  to  $86.7  million. 
The  total  DoD  advertising  budget  would  decline 
from  $242.9  million  to  $175.3  million.  The  Joint 
advertising  component  of  these  budgets  would  be 


maintained  at  $23.6  million,  and  $31.8  million 
respectively.  Hence,  the  Joint  advertising  share 
of  these  budgets  would  increase. 

Alternatively,  budget  implications  based  on 
the  analyses  presented  in  Chapter  7  indicate  a 
superior  level  of  system  performance  (measured 
by  applicant  rates)  than  either  the  direct  impli¬ 
cation  policy  or  current  budget  policy.  Based  on 
these  analyses,  several  different  mixes  and  levels 
of  advertising  expenditures  can  contribute  relative¬ 
ly  equally  to  fulfilling  recruiting  system  require¬ 
ments.  Depending  on  the  particular  combination 
of  budget  level  and  mix  chosen,  total  advertising 
spending  would  fall  while  Joint  advertising  would 
increase.  As  Table  8. 1  reflects,  the  FY  1987  DoD 
enlisted,  active-force  advertising  budget  would 
fall  from  $154.3  million  to  between  $92.2  and 
$1 12.5  million.  The  total  DoD  advertising  budget 
would  fall  from  $242.9  million  to  between 
$181.3  and  $201  million.  Here,  Joint  advertising 
components  of  these  budgets  would  rise 
substantially  from  $23.1  million  to  between  $35 
million  and  $41  million  and  from  $31.3  million  to 
between  $43  and  $50  million,  respectively. 

RECOMMENDATIONS 

Two  comments  should  be  made  regarding  the 
spirit  of  the  recommendations.  First,  no  single 
experiment  or  research  initiative  over  a  fixed 
period  of  time  provides  adequate  basis  for 
asserting  definitive,  deterministic  policy  mandates 
in  marketing.  Rather,  effective  application  of 
experimental  results  are  achieved  when  testing  is 
viewed  as  a  vehicle  that  facilitates  organized 
learning  about  the  limits  of  existing  marketing 
policies  and  about  the  feasibility  of  establishing 
better  policies.  The  recommendations  can  best  be 
viewed  as  suggestions  for  adaptive  initiatives 
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Table  8.1 


EXAMPLE  FY  1987  BUDGET 
CURRENT  AND  PROPOSED  LEVELS 


Millions  of  Dollars 


Current 

Proposed  Levels 

$4.35 
Per  Capita 
70%  Joint 

$5.00 

Per  Capita 
60%  Joint 

$5.50 

Per  Capita 

45%  Joint 

Total  DoD  Advertising  Budget 

242.9 

181.3 

191.9 

200.8 

Joint 

31.2 

48.9 

50.2 

432 

DoD  Enlisted  Active  Force 
Advertising  Budget 

154.3 

92.2 

103.3 

1122 

Joint 

23.1 

40.7 

42.0 

35.0 

DoD  NPS  Enlisted  Active  Force 
Advertising  Elements  Tested 

126.8 

65.7 

75.7 

84.6 

Joint 

22.7 

40.3 

41.6 

34.6 

based  on  the  results  of  the  experiment  This 
implies  an  ongoing  review  and  evaluation  of  these 
recommended  initiatives.* 


The  second  consideration  concerns  the  fact 
that  several  DoD  advertising  budgets  have  been 
proposed  and/or  approved  since  the  experiment 


*The  models  and  estimation  methodology  used  in 
this  study  are  consistent  with  those  of  previous 
research  on  factors  affecting  enlistment  supply. 
However,  more  recent  efforts  suggest  that  re¬ 
cruiter  behavior  variables  might  be  important  in 
manpower  supply  models  (Dertouzos,  1985; 
Carroll,  Lee  and  Rao,  1986).  Systematic 
changes  in  recruiter  behavior  can  alter  the  quan¬ 
tity  and  quality  of  enlistments  and  can  make 
estimating  the  impact  of  recruiting  resources, 
including  advertising,  difficult  To  the  extent 
that  changes  in  recruiter  behavior  are  correlated 
with  changes  in  advertising  expenditures,  the 
magnitude  of  the  advertising  effect  may  be  under¬ 
estimated. 

For  example,  a  sluggish  economy  or  other 
external  factors  could  produce  a  climate  in  which 


recruiters  would  have  little  trouble  meeting 
monthly  goals.  In  such  a  situation,  increases  in 
enlistment  supply  resulting  from  advertising  may 
not  result  in  observable  increases  in  actual  con¬ 
tracts.  The  advertising  effect,  in  the  absence  of 
suitable  recruiter  motivation,  may  merely  make  it 
easier  for  recruiters  to  achieve  their  objectives. 
Thus,  models  which  do  not  account  for  the  level 
of  recruiter  effort  may  not  capture  the  true  adver¬ 
tising  effect 

Accounting  for  such  factors  simultaneously 
for  all  four  Services  is  a  demanding  task  well 
beyond  the  scope  of  this  study.  Accordingly, 
any  effects  that  recruiter  behavior  variables  might 
have  had  on  the  findings  of  this  experiment  are 
unknown. 
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was  initiated  The  decisions  made  in  this  budget¬ 
ing  process  contrast  slightly  with  our  recommen¬ 
dations.*  More  specifically,  die  DoD  enlisted 
active-force  advertising  budget  has  grown  in  real 
terms  from  the  $108.7  million  FY  1984  budget 
level  to  the  proposed  FY  1987  level  of  $154.3 
million.  At  the  same  time,  the  proportion  of  the 
budget  allocated  to  Joint  advertising  has  decreased 
over  this  period  from  16.9  percent  to  15.2  per¬ 
cent  Although  the  results  of  die  experiment 
firmly  argue  for  budget  levels  moving  toward 
those  in  Cell  Blue  (lower  overall  expendi¬ 


ture  with  a  greater  proportion  allocated  to  Joint), 
exactly  the  opposite  has  occurred  during  the 
recent  budget  cycles. 

With  these  considerations  in  mind  three 
recommendations  based  on  the  DoD  Advertising 
Mix  Test  are  provided: 


1.  Budget  Policy  -  Reduce  DoD  recruitment 
advertising  budgets  while  increasing  the 
proportion  of  those  budgets  allocated  to 
Joint  advertising.  Both  the  direct  and 
alternate  budget  implications  of  the  test 


*Application  of  Research  Findings 

In  July  1986,  the  Deputy  Secretary  of  Defense 
reviewed  the  findings  of  the  DoD  Advertising 


goal  to  achieve  a  25-percent  reduction  in  the  total 
DoD  advertising  budget  by  FY  1991.  Table  8.2 
ivs  the  target  advertising  budget  levels  for 


reviewed  the  findings  of  the  DoD  Advertising 
Mix  Test.  After  full  consideration  of  the  findings 
and  the  recruiting  environment,  he  decided  to 
phase  in  reductions  to  total  DoD  advertising  and 
to  effect  cost  savings  by  reducing  Service  and 
slightly  increasing  the  Joint  advertising  budgets. 
Specifically,  the  Deputy  Secretary  established  a 


displays  the  tar® 
the  Services  ana 


ertising  budget  levels  for 


the  Services  and  the  Joint  program  for  FYs  1988 
through  1992. 

Continuous  monitoring  of  Service  recruiting 
performance  will  be  conducted  to  ensure  that 
adequate  resources  are  provided  to  support  DoD 
recruiting  efforts  and  thatjustifiable  annual  adver¬ 
tising  budgets  can  be  formulated  and  defended. 


Table  &2 

DoD  Recruitment  Advertising  Budgets 
FY  1988  -  FY  1992 
(Current  Year  Dollars  in  Millions) 


FY  88 

FY  89 

FY  90 

Em 

FY  92 

Army 

124.8 

120.2 

114.6 

108.1 

110.6 

Navy 

36.2 

25.7 

24.3 

22.6 

23.1 

Marine  Corps 

18.8 

17.7 

16.5 

15.0 

15.3 

Air  Force 

23.3 

22.8 

22.1 

21.2 

21.9 

Joint 

33.5 

35.0 

36.4 

37.7 

38.6 

TOTAL 

236.6 

221.5 

213.9 

204.6 

209.5 

ft 


ft 


•v 

w 


K 


g 


nWSR'SM 


iw:v 


lead  to  this  conclusion.  While  the  level 
and  mix  of  advertising  spending  repre¬ 
sented  in  Cell  Blue  produced  the  best 
recruiting  system  performance  of  the 
spending  plans  tested  in  the  experiment. 
Cell  Blue  does  not  represent  the  best  or 
optimum  mix  at  that  level  of  total  spending 
to  maximize  recruiting  system  perfor¬ 
mance.  The  analyses  in  Chapter  7  indicate 
that  a  higher  level  of  recruiting  system 
performance  can  be  achieved  by  taking 
into  account  the  effect  on  recruiting  of  the 
observed  interaction  between  the  total 
advertising  budget  level  and  its  associated 
mix  of  Joint  and  Service-specific  adver¬ 
tising. 

Table  8.1,  for  example,  depicts  budgets 
which  take  into  account  this  interaction 
effect  in  establishing  the  appropriate  mix 
of  Joint  and  Service-specific  advertising  at 
alternative  total  spending  levels.  The 
analysis  in  Chapter  7  that  served  to 
quantify  this  interaction  effect  also 
reinforces  the  conclusion  derived  from 
analyses  of  the  test  cell's  spending  plans 


that  overall  advertising  budgets  are  too 
large  and  that,  as  the  total  budget  is 
reduced,  a  corresponding  increase  in  the 
proportion  allocated  to  Joint  advertising  is 
clearly  warranted. 

2.  Market  Testing  -  Provide  some  markets 
with  advertising  budget  levels  both  signi¬ 
ficantly  higher  and  lower  than  the  levels 
recommended  here.  This  allows  system¬ 
atic  learning  about  the  adaptation  of  the 
system  to  the  new  budget  levels  and  about 
any  longer-term  impacts  of  the  recom¬ 
mended  budget  levels. 

3.  Research  -  Considerable  resources  will 
continue  to  be  invested  in  recruitment 
advertising.  The  effectiveness  of  these  in¬ 
vestments  should  continue  to  be  assessed. 
More  specifically,  continued  research  to 
understand  better  the  relationship  between 
advertising  expenditures  and  enlistment 
intentions  as  well  as  between  measures  of 
enlistment  intentions  and  actual  enlistment 
behavior  is  recommended. 
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ASSISTANT  SECRETARY  OF  DEFENSE 


WASHINGTON.  D.C.  20101 


MANTOWCN. 
ItCStftVC  ATTAINS 
A  NO  LOGISTICS 


8  JUL  1983 


MEMORANDUM  FOR  THE  ASSISTANT  SECRETARY  OF  THE  ARMY  (MARA) 

ASSISTANT  SECRETARY  OF  THE  NAVY  (MARA) 
ASSISTANT  SECRETARY  OF  THE  AIR  FORCE  (MRAAI) 


SUBJECT:  FY84  Advertising  Mix  Test  Concept  Design 


Attsched  (Tsb  A)  is  the  Approved  concept  design  for  the  Advertising  Mix  Test 
which  begins  October  1(  1983.  As  the  discussion  of  the  issues  st  Tsb  B 
shows,  we  generally  agreed  with  your  comments  and  racowndotlons  on  the  draft 
test  design  which  we  staffed  In  May  of  this  year. 


I  want  to  thank  you  and  your  staffs  for  the  continued  effort,  cooperation 
and  comments  on  this  Important  project. 


Lawrence  j.  Kerb 
Assistant  Secretary  of  Defense 
(Manpower,  Reserve  Affairs  £  Logistics) 
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ADVERTISING  TEST  CONCEPT  DESIGN 


I.  BACKGROUND 

For  the  pate  few  years.  Congress  has  been  concerned  with  the  level  of  DoD 
recruitment  advertising  expenditures.  The  Secretary  of  Defense  has 
posed  legitimate  management  questions  as  to  the  cost  effectiveness  of 
the  mix  of  Joint  Service  and  Service-specific  advertising.  Since  the 
Department  of  Defense  does  not  have  a  methodology  which  relates  and 
quantifies  the  effect  of  varying  levels  of  advertising  to  actual  enlistments 
responding  to  these  policy  questions  has  been  difficult. 

In  FT  1981,  the  Congressional  Budget  Office  (CBO)  recommended  significant 
increases  in  the  Joint  Recruiting  Advertising  Program  (JRAP)  with  concurrent 
reductions  in  Service-specific  advertising  and  a  net  overall  savings. 

The  Secretary  of  Defense  approved  the  CBO  concept  of  increasing  the 
Joint  Service  advertising  program  but  not  at  the  amounts  suggested  by 
the  CBO.  His  guidance  was  to  double  the  size  of  the  Joint  Service  program 
in  FT  1982  while  reducing  the  Service-specific  and  net  DoD  advertising 
costs. 

Quite  properly,  the  individual  Services  are  concerned  that  cutbacks  in 
Service-specific  Sdvertlslng  will  adversely  effect  their  ability  to  meet 
recruitment  goals  and  to  sustain  the  gains  made  in  quality  in  the  last 
two  years.  They  believe  that  Joint  advertising* s  role  is  one  of  "corporate** 
or  umbrella  advertising  which  complements  the  main  thrust  of  the  "product” 
advertising  of  the  individual  Services. 
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During  the  summer  of  FY  1981,  -the  Secretary  of  Defense  continued  the  dialogue 
about  the  proper  Service-specific/ Joint  Service  advertlalng  six  and  the  most 
appropriate  total  advertising  program  levels.  After  such  discussion,  the 
Secretary  decided  to  hold  the  advertising  nix  and  total  funding  at  current 
levels,  and  to  conduct  a  major  field  test  to  determine  the  most  appropriate 
levels  and  mix  of  advertising  funding. 
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An  outside  contractor  was  retained  in  September  1982  to  design  an  in-market 


test  which  would  generate  quantitative  data  to  help  answer  the  question: 
"What  is  the  optimum  mix  of  Jolnt/Service-speclfic  recruitment  advertising 
for  achieving  Active  Enlisted  N?S  goals  at  different  levels  of  total  DoD 
recruitment  advertising?"  The  test  would,  therefore,  entail  testing  two 
variables  simultaneously: 


a.  The  level  of  the  total  DoD  Active  Enlisted  NFS  Recruitment 
Advertising  budget  — •  which  includes  both  Joint  and  Service- 
specific  advertising;  and 

b.  The  mix  of  Joint  and  Service-specific  advertising  within 
each  total  advertising  expenditure  level. 

In  particular,  the  test  would  assess  the  hypothesis  that  the  same 
recruitment  effectiveness  can  be  achieved  by  increasing  the  Joint 
budget  significantly  while  decreasing  the  overall  DoD  Active  Enlisted 
NFS  Recruitment  advertising  budget. 


It  is  also  expected  that  such  a  test  would  develop  information  which 
would  provide  a  better  understanding  of  the  relationships  smong 


recruitment  advertising  and  awareness  of  the  individual  Services  and 
the  benefits  they  offer;  attitudes  toward  the  Services;  and  various 
measures  of  recruitment  success,  including  applicants,  contracts,  and 
accessions. 
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An  initial  experimental  design  was  developed  In  Che  Fall  of  1982  which 
consisted  of  nine  cells*  While  conceptually  sound,  this  design  was 
Judged  to  be  too  complex,  costly  and  disruptive  to  the  recrultaent  pro- 


grans.  Based  on  this  conclusion,  a  four-cell  test  design  was  suggested 
which  was  scientifically  sound  and  operationally  acceptable  to  the  Ser¬ 
vices*  Specific  infomatlon  about  the  four-cell  test  design  is  presented 
In  this  paper. 

II*  METHODOLOGY  COHSIDEKATIONS  AND  ASSUMPTIONS 

Six  considerations  and  aasunptlons  were  taken  Into  account  In  designing 
the  test. 

A.  Definition  of  Variables  to  be  Tested 

As  designed  this  test  Is  concerned  only  with  the  Active  Enlisted 
Hon-Prlor  Service  recrultaent  advertising.  Further,  It  Is  concerned 
solely  with  the  media  advertising  funds  as  reported  to  Congress. 

A  substantial  mount  of  the  total  advertising  budget  Is  for  recruiting 
Reserves,  nodical  scholarship  progress ,  officers  and  ROTC  scholarship 
program.  Moreover,  there  are  a  number  of  support  advertising  program 
Included  In  the  advertising  budget  such  as:  lead  fulfillment , 
sales  promotion,  market  research,  and  print  material  and  literature. 
When  these  program  are  excluded  from  the  total  budget,  the  Active 
Enlisted  HourPrlor  Service  Recruitment  Advertising  expenditures  for 
FI  1982  constituted  only  583. 3  million  of  the  total  5155  ailllon. 

The  Service  expenditures  for  Active  Enlisted  Non-Prior  Service  adver¬ 
tising  Is  shown  In  Table  1. 


Table  1 

FT  1982 (Actual)  Advertising  Allocations 
($  In  Thousands) 


$45,412 


Marines 

8,000 

Air  Force 

4,305 

Subtotal 

Service-Specific 

-  $67,670 

Joint 

15,831 

Total 

-  $83,501 

Accordingly,  these  FY  1982  figures  will  be  the  base  expenditures  and 
the  base  alx  around  which  total  expenditures  and  the  Joint /Service- 
specific  mix  can  be  varied.  Fund  allocation  for  each  Service  Is  based 
on  the  percentage  spent  by  that  service  of  the  total  actual  FT  1982 
advertising  funds  without  adjustment  for  inflation. 

Bepresentat 1 vanes s  and  Control  of  Test  from  Effects  of  Extraneous  Variables 


In  order  for  the  test  to  be  valid,  DoD  specified  that  each  cell  In  the 
test  must  be  representative  of  the  total  0.S.  Further,  the  design  and 
control  of  the  test  has  to  Insure  that  extraneous  variables  will  not 
confound  the  results. 

In  addition  to  advertising,  there  are  many  variables  which  affect 
recruitment  success.  Included  in  this  list  of  variables  are:  the  size 
of  the  population.  Service  enlistment  rates,  minority  population, 
unemployment  rates,  propensity  to  enlist,  geographic  regions  and  the 
slse  of  the  market. 
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The  Cate  must  b«  balanced  so  that  tha  influence  of  thasa  variables  is 


eoatrollad;  that  is,  thay  af fact  aach  group  of  tast  markets  equally. 

Under  thasa  conditions,  tha  only  Independent  variable  which  will  be 
different  from  one  group  of  test  markets  to  another  will  be  the 
total  Active  Enlisted  NFS  advertising  level  and  the  Joint/Servlce- 
spedflc  mix.  Therefore,  any  differences  observed  in  the  recruitment 
process  would  be  attributable  to  the  advertising  expenditure  level 
or  Joint/ Service-specific  mix  and  not  to  other  variables. 

A  basic  requirement  la  that  each  group  of  test  markets  be  representative 
of  the  recruitment  conditions  found  throughout  the  country.  In  effect, 
each  group  of  test  markets  will  be  a  microcosm  of  the  D.S.  At  the  same 
time,  each  group  of  markets  must  be  Independent  from  a  media  point  of 
view,  so  that  advertising  placed  in  one  group  of  markets  does  not  spill 
over  into  any  other  group  of  test  markets  to  any  significant  degree. 

Perhaps  more  importantly,  a  country  the  size  of  the  D.S.  is  not  homogen¬ 
eous.  Thus,  co  pick  a  group  that  represents  the  heterogeneity,  l.e.,  the 
tremendous  variation  from  place  to  place,  of  the  country,  a  sufficient 
number  of  markets  must  be  chosen  to  offset  the  nonhoaogeneity  characteristics 
Time  Span  of  the  Test  to  Cover  Cyclical  Recruitment  Environments 
It  is  generally  agreed  that  it  takes  time  for  advertising  to  work. 

Although  it  may  begin  to  work  immediately,  its  effects  are  not  apparent 
or  measureable  for  at  least  several  months.  Its  full  effects  are  not  felt 
for  perhaps  as  long  as  one  to  two  years. 

Therefore,  the  test  must  be  operative  for  a  long  enough  period  of 
time  to  permit  the  full  effects  of  advertising  to  have  impact. 


Another  factor  which  affects  both  the  duration  of  the  test  and  the 
criteria  of  evaluation  Is  the  economic  eovironsMnt.  In  periods  of 
high  unemployment,  recruitment  Is  easier,  particularly  aaong  the 
17-22  age  group.  During  such  tines,  economic  motivations  become 
the  dominant  factor  and  could  easily  obscure  the  effects  of  advertising 
If  the  economic  environment  remained  at  a  conatant  level  (l.e.,  high 
unemployment)  throughout  the  test  period.  It  may  not  be  possible  to 
measure  any  effects  of  variations  In  expenditures  levels  and/or 
advertising  mix,  at  least  not  In  behavioral  terns  (l.e.,  applicants, 
contracts  or  accessions). 

This  has  Implications  for  the  test  design  that  must  be  taken  into 
consideration: 

The  test  must  be  staged  long  enough  to  antldpete  changes  In  the 
economic  environment  from  a  high  unemployment  to  a  low  unemploy¬ 
ment  situation  to  permit  the  varying  effects  of  advertising  to 
be  observed.  Accordingly,  OSD  will  decide  in  Spring  84  whether 
to  extend  the  test  for  an  additional  year. 

Both  the  attltudinal  and  behavioral  criteria  should  be  Included 
In  the  test  In  order  to  measure  the  full  effects  of  advertising. 

Relevance  of  Test  Measurements 
A  number  of  measurements  ere  available: 

Applicants.  (Individuals  who  have  taken  the  production  ASVAB) 
Contracts. 

Accessions,  (special  emphasis  on  CAT  1-111A  High  School  Graduates) 
Awareness  and  propensity,  (of  military-aged  youth  population) 


The  ability  of  these  measurements  to  vary  with  respect  to  changes  In 
advertising  level  and  Joint/Servlce-speclflc  mix  may  depend  upon  the 
specific  recruitment  environment. 


Needless  to  say,  for  the  test  to  be  relevant  and  meaningful,  the  yardsticks 
by  which  the  results  are  assessed  should  be  related  to  the  recruitment 
process.  Every  attempt  must  be  made  to  continue  to  identify/develop 
new,  cost-effective  measures. 

Potential  Impairment  of  Service  Recruitment  Results 

It  is  important  that  the  test  not  unduly  interfere  with  the  individual 

Services  achieving  their  overall  recruiting  goals. 

The  test  entails  decreasing  the  amount  of  Service-specific  advertising 
expenditures  in  some  of  the  test  markets.  There  is  obviously  some  risk 
involved  in  this  procedure  since  advertising  may  be  the  key  element  in 
maintaining  Service  awareness  and  propensity  to  enlist.  Awareness 
and  propensity  to  enlist  could  decay  over  time  with  continued  low 
levels  of  advertising.  This  is  particularly  critical  based  on  the  extent 
to  which  awareness  and  propensity  to  enlist  are  related  to  applications 
and  contracts. 

Again  this  hai  two  implications  for  the  test  design: 

The  allocation  of  Service-specific  funds,  on  the  downside,  in  the 
test  markets  must  be  made  in  such  a  way  as  to  entail  as  little  risk 
as  possible  to  the  Services'  long  term  awareness  and  propensity  to 
enlist. 

The  duration  of  the  test  must  allow  for  obtaining  measurable 
results,  as  well  as  minimizing  the  risk  of  decay  in  Service 
awareness  over  the  long  term. 

A- 8 
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wealth  of  demographic  and  socio-economic  data  collected  at  the  ADI  level 
of  detail. 


The  control  cell  (Cell  White)  conalate  of  7 62  of  the  military- aged  youth  pop¬ 
ulation  of  the  country.  In  thia  cell,  which  includes  the  two  largest  AD Is — 
New  York  and  Los  Angeles — the  advertising  levels  will  mirror  those  in  effect 
in  FT  1982.  Thus,  for  the  control  cell  the  national  rate  of  $83.5  million 
in  total  military  advertising  will  be  simulated.  This  breaks  down  to  a 
national  rate  of  $15.8  million  for  Joint  and  $67.7  million  for  Service-specific 
advertising.  The  total  annual  advertising  funds  for  the  control  cell 
will,  of  course,  be  lower  than  the  $83.5  million  rate  expanded  nationwide. 

In  fact,  the  actual  funding  will  be  762  of  the  national  level  of  $83.5  million 
or  $63.5  million  for  Cell  White. 


The  by-service  allocation  for  this  cell  is  shown  in  Teble  2. 


Table  2 


Annual  Advertising  Funding  for  the  Control  Cell  Based  on  Actual  FY  82  dollars 

($  in  Thousands) 


V' A 


The  second  test  cell  (Cell  Blue)  consists  of  8Z  of  the  country  or  14  ADIs. 
This  c«ll  will  include  the  current  nstionel  rate  of  Joint  advertising 
($15.8  Billion)  and  a  greatly  reduced  Service-specific  level  (a  $14.7  million 
national  rate).  The  actual  funds  for  this  cell  are  $1.27  alllion  for  Joint 
and  $1.17  Billion  for  Service  or  a  total  cost  of  $2.4  Billion  for  the  cell. 
The  Service-specific  amount  is  based  on  the  FT  1982  national  rate  of  local 
advertising  which  was  viewed  as  the  alnlaua  level  for  having  a  service  progra 
The  by- Service  aaounts  for  this  cell  are  shown  below  in  Table  3. 


Table  3 

Annual  Advertising  Funding  for  Test  Cell  Blue  (FT  82  dollars) 

($  in  Thousands) 


$  2,443 


The  third  test  cell  (Cell  Green)  which  is  composed  of  8Z  of  the  nation  and 
10  ADIs  consists  of  a  auch-decreascd  Joint  advertising  level  ($4m  national 
rate)  and  a  Service-specific  nation  rate  equal  to  FT  1982  levels  ($67. 7M). 

The  actual  costs  for  this  cell  are  $.3  million  for  Joint  and  $5.4  Billion  for 
Service  or  a  total  cost  of  $5.7  Billion  for  the  cell.  The  allocation  for  this 


cell  follows: 
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Annual  Advert la inf  Funding  for  Taat  Call  Craan  (FT  82  dollara) 

($  in  Thouaanda) 


National 

Actual 

Rate 

Funding 

Army 

$43,412 

$  3,633 

Navy 

9,953 

796 

Marine  Corps 

8,000 

640 

Air  Force 

4,305 

344 

Joint 

4.000 

320 

Tha  fourth  taat  call  (Call  Rad)  consists  of  a  much  largar  Joint  advertising 
prograa  and  a  raducad  Sarvlca-spacif lc  total.  This  taat  cell,  which 
again  reprasants  8X  of  tha  nation  and  18  AOl's,  provides  for  a  national 
rate  of  $40M  for  Joint  advertising  and  combined  Service  specific  advertising 
levels  of  $14.7  nllllon.  Tha  actual  total  funding  for  this  cell  la  $3.2  Bil¬ 
lion  for  Joint  and  $1.2  nllllon  for  tha  coabinad  Service-specific.  The  ration 
ale  for  the  Service-specific  levels  in  this  cell  are  the  sane  as  those  In 
test  Cell  Blue.  The  $40  nllllon  Joint  level  Is  based  on  a  previously  used, 
although  not  uniformly  accepted,  formula  which  coapares  the  Increases 
needed  In  Joint  components  for  the  prescripted  decreases  In  the  Service- 
specific  prograa.  The  allocation  anong  the  Services  follows: 

Table  5 

Annual  Advertising  Funding  for  Test  Cell  Rad  (FT  82  dollars) 

($  In  Thousands) 


National 

Actual 

Rate 

Funding 

Army 

$  7,139 

$  571 

Navy 

2,535 

203 

Marine  Corps 

2,721 

218 

Air  Force 

2,317 

185 

Joint 

40.000 

3,200 

-<*  a«J  .11  *14  .1^11  .1*  .11  .!»  .’*  i 


A  ■  usury  of  Cho  Cost  funding  by  Service  ie  shown  in  Che  following  cable: 

Table  6 

Susaary  TesC  Cose  by  Services  and  Cell* 

($  in  Millions) 


Control 

Test 

Test 

Test 

National  Rate 

Cell 

Cell  Blue 

Cell  Green 

Cell  Red 

Army 

$45.4 

$  7.1 

$45.4 

$  7.1 

Navy 

10.0 

2.5 

10.0 

2.5 

Marine  Corps 

8.0 

2.7 

8.0 

2.7 

Air  Force 

4.3 

2.3 

4.3 

2.3 

Joint 

15.8 

15.8 

4.0 

40.0 

Total 

$3375 

$3CT 

$70 

$54.7 

Actual  Funding 

Army 

$34.5 

$  .6 

$  3.6 

$  .6 

Navy 

7.6 

.2 

.8 

.2 

Marine  Corps 

6.1 

.2 

.6 

.2 

Air  Force 

3.3 

.2 

.3 

.2 

Joint 

12.0 

1.3 

.3 

3.2 

Total 

$30 

$  2.4 

$  377 

$  4.4 

Toeal  Pro 


*  Numbers  say  noc  cocal  due  Co  rounding 


A  suaaary  of  che  cesc  design  is  provided  in  Che  following  two  charts. 

Chart  A  shows  the  national  races  for  Joint  and  Service-specific  (in  lower 
left  corner  of  each  cell),  the  percent  of  Che  country  in  che  cell  and 
Che  number  of  ADIs  in  the  cell.  Additionally,  Chart  B  displays  simulated 
as  well  as  actual  allocations  per  cell  for  Joint  vs  Service-specific 
advertising. 

The  proposed  allocation  of  ADIs  among  che  cells  is  shown  in  Chart  C. 

The  design  was  created  by  randomised  assignment  of  ADIs  Co  che  cest 
programs,  subject  to  constraints  stipulating  that  che  means  of  the  variables 
be  closely  matched  across  Che  four  test  cells.  This  is  Che  best  balance 
of  ADIs  on  the  variable  of  propensity. 
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ADVERTISING  TEST 


Sot*:  la  **ch  of  th*  four  colls,  th*  Joint  sad  S*rvlc*-sp*elfle  Mounts  sr*  simulated 
advertising  «xp*nditur*s  sad  not  actual  expenditures. 


CONTROL  CELL! 

CELL  Whit*  (76Z  of  th*  nation) 


168  AD Is 
JOINT  —  SAME 

SERVICE  SPECIFIC  —  SAME  (CUUENT) 

Joint  $  15. 8M/ Service  Sp*dfic  $67. 7M 


cell  Gr*«n  (8Z  of  th*  nation) 
10  ADls 

LOWES  JOINT 

COSBENT  SESVICE  SPEaFIC 

Joint  $4m/S*rvlc*  Specific  $67. 7M 


•TO 


CELL  Blu*  (8Z  of  nation) 

14  ADIs 
SAME  JOINT 

LOWES  SESVICE  SPECIFIC 

Joint  $15.8M/S*rvic*  Specific  $14. 7M 


w' ’  V 

i 


CELL  Sad  (8Z  of  th*  nation) 

18  ADIs 
HIGHER  JOINT 
LOWES  SESVICE  SPEaFIC 

Joint  $40n/S*rvlc*  Specific  $14. 7M 


c*5w_ 

f 


M 


CHART  A 


ADVERTISING  TEST 


CONTROL  CELL 

CELL  White  (76Z  of  Che  nation) 

CELL  Blue  (8Z  of  nation) 

JOINT  —  AS  DESIRED  (CURRENT) 

SAME  JOINT 

SERVICE  SPECIFIC  —  SAME  (CURRENT) 

($  In  Millions) 

LOWER  SERVICE  SPECIFIC 
($  in  Millions) 

i 

1 

Joint  Service 

Total 

Joint 

Service 

Total 

Simulated  $15.8  $67.7 

Actual  $12.0  $51.4 

$83.5 

$63.4 

Simulated  $15.8 
Actual  $  1.3 

$14.7 
$  1.2 

$30.5 
$  2.4 

CELL  Greta  (8Z  of  the  nation) 

CELL  Red  (8Z  of 

the  nation) 

LOWER  JOINT 

HIGHER  JOINT 

CURRENT  SERVICE  SPECIFIC 
($  In  Millions) 

LOWER  SERVICE  SPECIFIC 
($  In  Millions) 

Joint  Service 

Total 

Joint 

Service 

Total 

Simulated  $  4.0  $67.7 

Actual  $  .3  $  5.4 

$71.7 
$  5.7 

Simulated  $40.0 
Actual  $  3.2 

$14.7 
$  1.2 

$54.7 
$  4.4 

CHART  B 


ADVERTISING  MIX  TEST  DESIGN 


Test  Measurements 


The  three  pre-identlfied  measures  of  effectiveness  by  which  this  test 
will  be  evaluated  are: 

Change  in  propensity  levels,  awareness  and  attitudes. 

The  change  in  the  number  of  applicants. 

The  change  in  the  number  of  contracts. 

Information  on  applicants  and  contracts  is  available  from  records  maintained 
by  the  Services.  Propensity,  awareness  and  attitudes  are  available 
from  TATS,  although  somewhat  limited  due  to  current  sample  sixes. 

It  is  generally  agreed  that  additional  attltudlnal  measures  would 
greatly  enhance  the  test  measurement.  However,  the  cost  of  gathering 
this  information  at  required  levels  of  statistical  precision  is  very 
high.  Thus,  further  trade-off  analysis  and  thought  are  necessary. 

If  feasible,  this  additional  attltudlnal  information  will  have  to  be 
collected  in  FT  1984  in  a  special  survey  of  applicants. 

All  measurements  will  be  studied  in  aggregated  and  disaggregated  form 
so  as  to  make  sure  that  quality  goals  are  properly  assessed. 

Data  and  Planning  Requirements 

The  test  requires  the  preparation  of  an  advertising  plan  for 
each  cell  by  jiach  of  the  Services.  In  other  words,  each  Service 
must  prepare  four  advertising  plans.  The  only  constraint  on  these 
plans  is  that  no  major  creative  strategy  changes  should  be  made  for 
the  duration  of  the  test,  although  specific  executions  can  be  changed. 
Similarly,  an  advertising  plan  must  be  prepared  for  using  Joint 
advertising  funds  in  each  cell  in  as  optimum  a  manner  as  possible. 


A  large  number  of  pbanoaana  other  then  advert ielng  will  be  observed 
In  treatment  narfceta  during  the  course  of  the  experimental  intervention. 
Soatt  will  have  significant  effects  on  recruitment  performance  and  must  be 
explicitly  considered  in  the  analysis  of  the  experiment.  These  must 
be  specified,  collected,  validated  and  dlsagregated. 

A  detailed  data  plan  Including  the  data  definition,  formats,  frequencies 
and  other  specifications  will  be  staffed  prior  to  the  beginning  of  the 
teat.  It  is  envisioned  that  data  to  be  reported  will  include,  but  not 
be  limited  to: 


1.  By  Market  (by  ADI) 

Youth  population 

Segmentation  (male,  female,  minority,  quality,  etc.) 
Household  Income 

Incidences  of  military  Installations 
Economic  and  employment  conditions 
Recruitment  goals 

Historical  recruitment  performance 

Projected  recruitment  performance  (test  period) 

DEP  posture 


2.  Resources  (by  Cell) 

Production  recruiter  strength  (manning) 

Advertising  expenditures  by  node,  by  month  (national,  local,  etc.) 
Advertising  expenditures  by  medium,  by  month 
Impressions  by  node 
Impressions  by  medium 
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3.  Performs nee  (by  Cell) 
Measured  awareness 


Accltude  shifts 
Prospect  leads 
4.  Activity  (by  Cell) 

Applicants 

Contracts 

Finally,  certain  monitoring  mechanisms  will  be  required  for  the  duration 
of  the  test  with  regard  to  advertising  and  recruitment  activity  related 
to  the  experimental  intervention  to  ensure  validity  and  unambiguity. 
Monitoring  mechanisms  and  tolerances  such  as  total  advertising  expendi¬ 
tures  and  delivery,  (including  such  items  as  reserve  and  officer  programs 
expenditures),  recruitment  gosling  and  production  activity,  production 
recruiter  strength  and  manning,  recruiter  production  incentive  plans, 
broad  recruitment  policy  and  other  policy  constraints  must  be  specified 
prior  to  the  beginning  of  test.  Vlthout  monitoring  mechanisms  it 
would  be  difficult  to  analyze  the  specific  effects  of  advertising. 

DoD  is  committed  to  ensuring  adherence  to  agreed  controls.  All  partici¬ 
pants  in  the  experimental  intervention  will  monitor  their  units,  activities 
and  commercial  advertising  agencies  for  compliance  and  will  report  ex cur- 
slons  and  magnitudes  to  the  designated  OoO  polnt-of-contact  as  they  occur. 
DoD  will  access  the  impact  of  these  items  and  arbitrate  as  required. 


ft 
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D.  Tlaetable 

The  CMt  will  begin  in  October  1983  end  lent  for  12  aonths  with  en 

additional  12  aonth  period  if  necessary.  To  accoaplish  this  start 

date  the  following  allestones  are  required  and  aust  be  act. 

Approval  of  test  concept  design  July  1983 

Detailed  plan  of  data/ Inf oraatlon/ Controls  (draft)  July  1983 

Developaent  of  Service  &  Joint  August  1983 

test  plans  (by  test  cell) 

Base  line  data  collection/  Sc p tea  be r  1983 

attltudlnal  lnforaation 


Test  begins 


October  1983 


Table 


Ap  pa  ad  lx  A 
Balancing  cha  ADIa 


1.  Advertising  Mix  Tast  Calls 

2.  Balancing  Varlablas 

3.  Susan ry  of  Design 

4.  list  of  ADls 

*  Call  White 

*  Call  Blua 

*  Call  Graan 

*  Call  Bad 
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Table  1 


ADVERTISING  MIX  TEST  CELLS 


Advertising 
(SMillion  per  Tear)* 


Cell 

Prograa 

Joint 

Advertising 

Service- 

Specific 

Advertising 

Percent  of 
Nation  In 
Cell 

White 

Control 

(7Y82  Advertising  Level) 

15.8 

67.7 

76 

Blue 

Reduced  Service  Advertising 

15.8 

14.7 

8 

Green 

Reduced  Joint  Advertising 

4 

67.7 

8 

Red 

Increased  Joint  Advertising 

40 

14.7 

8 

‘Expenditure  rate  necessary  for  e  natioo-vlde  program  During  the  test, 
this  rate  will  be  pro-rated  to  the  size  of  population  In  the  test  cell. 
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Table  2 


BALANCING  VARIABLES 
Advertising  Mix  Test  Design 


Variable  Name 

Abbreviation 

Definition 

Source 

Size  of  Population 

SIZE* 

Number  of  males  age  17-21 

DMDC  estimate 

in  the  area,  1982 

from  Census 

data 

Enlistment  Rates 

Army 

ARMY 

Number  of  male  high-quality 

DMDC  data 

Navy 

NAVY 

enlistments  (high  school 

from  MEPCOM 

Air  Force 

AF 

diploma  graduates  and 

ARS  system 

Marine  Corps 

MC 

seniors  with  AFQT  scores 

OoD 

DOD 

of  SO  or  above) ,  calendar 
year  1982,  as  a  percent 
of  1982  male  population 
age  17-21 

Nonwhite  Population 

NONW 

Black  and  Hispanic  males 
age  17-21,  as  a  percent  of 
total  male  population  age 
17-21,  1982 

Census 

Unemployment  Rate 

UNEM 

Unemployment  rate  in 

Bureau  of 

manufacturing,  all  ages. 

Labor 

calendar  1982 

Statistics, 
county  file 

Propensity  to  Enlist 

PROP 

Percent  of  area  survey 

1982  Youth 

respondents  who  reported 

Attitude 

positive  propensity  to 

Tracking 

enlist  ("definite"  or 

Study,  MEPS 

"probable"  intention) 

level  data 

Geographic  Region 

Percent  of  1982  area 

Census 

Northeast 

NE 

population  (males  17-21) 

Vest 

V 

living  in  each  major 

South 

S 

geographic  region 

Market  Size 

TOP  12 

Percent  of  area  1982  popu¬ 
lation  (males  17-21)  living 
in  one  of  the  12  most  popu¬ 
lous  AD Is  (which  make  up 
one- third  of  the  U.S. 

Census 

population) 

Cells  are  not  actually  balanced  on  this  variable.  It  is  used  to  define 
the  size  of  the  test  cell  and  as  an  eleaent  in  conputation  of  the  other 
variables. 
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APPENDIX  B 


RAND  LETTER 

METHODOLOGY  FOR  ASSIGNING  MARKETS  TO  TEST  CELLS 


APPENDIX  B 


22  April  1983 


Dr.  G.  Thomas  Sicilia 
Director  of  Accession  Policy 
Office,  Assistant  Secretary  of 
Defense  (Manpower,  Reserve 
Affairs,  and  Logistics) 

U.S.  Department  of  Defense 
Washington,  DC  20301 


Dear  Tom: 


Enclosed  are  documents  detailing  two  design  options  for  the  DoD  Advertising 
Mix  test  (Attachments  1  and  2) .  Both  options  were  created  using  the 
methodology  for  experimental  design  which  we  developed  for  the  Enlistment 
Bonus  Test. 


Table  1  outlines  the  basic  assumptions  under  which  we  prepared  the  designs. 
We  assume  that  the  test  will  compare  three  new  advertising  programs  with  a 
control  program.  Each  new  program  will  be  implemented  in  a  "test  cell"  (a 
set  of  local  areas)  comprising  8  percent  of  the  U.S.  population.  The  con¬ 
trol  program  will  be  implemented  in  the  remaining  areas,  containing  76 
percent  of  the  population.  In  preparing  design  options,  our  purpose  was 
to  assign  local  areas  to  the  four  cells  so  as  to  balance  the  cells  on 
factors  that  may  affect  enlistments.  The  factors  we  considered  are  defined 
in  Table  2. 


For  advertising  purposes,  local  areas  are  defined  by  Areas  of  Dominant 
Influence  (ADIs),  which  are  groups  of  counties  that  are  mutually  exclusive 
and  that  jointly  include  all  of  the  continental  United  States.  ADIs  are 
designated  by  the  Arbitron  Company  according  to  predominant  local  tele¬ 
vision  viewing  patterns.  However,  in  some  localities  there  is  substantial 
"spill-in"  of  stations  from  other  ADIs.  For  example,  television  stations 
from  Boston  spill  into  the  Providence,  Rhode  Island  ADI,  accounting  for  an 
estimated  27  percent  of  the  viewing  hours  in  Providence.  To  minimize  this 
spill-in,  which  would  dilute  the  test  programs'  effects,  we  linked  ADIs 
together  in  the  assignment  procedure  whenever  one  of  them  accounted  for 
more  than  15  percent  of  viewing  hours  in  another's  market.  Lists  of  the 
ADIs  and  the  linkages  are  given  in  Attachments  3  and  4. 


The  designs  were  created  by  randomized  assignment  of  ADIs  to  the  test 
programs,  subject  to  constraints  stipulating  that  the  means  of  certain 
variables  be  closely  matched  across  the  four  test  cells.  As  listed  in 
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hr. 


Dt.  G.  Thomas  Sicilia 
22  April  1983 


Table  2,  we  imposed  constraints  on  (1)  enlistment  rates  for  each  of  the 
four  military  services  and  DoD;  (2)  minority  population;  (3)  unemploy¬ 
ment  rate;  (4)  propensity  (intention)  to  enlist,  as  expressed  in  sur¬ 
veys;  (S)  geographic  region;  and  (6)  population  in  large,  medium,  and 
small  sized  ADIs.  Among  several  designs  randomly  generated  by  our 
procedures,  we  selected  the  two  with  the  best  balance,  considering  the 
precision  of  the  matching,  the  degree  of  geographic  dispersion  of  the 
ADIs  in  each  cell,  and  the  effects  of  imbalances  on  the  variances  of 
statistical  contrasts  between  the  test  cells.  The  statistical  proce¬ 
dures  are  those  outlined  in  our  recent  Note  on  experimental  design  for 
the  bonus  test.* 


Table  3  shows  a  summary  of  the  cell  characteristics  for  each  design. 

Both  designs  are  fairly  well  matched  on  most  of  the  balancing  varia¬ 
bles.  We  recommend  Option  1  because  it  is  better  matched  on  enlistment 
rates  and  because  it  has  better  geographic  dispersion  across  the  nation. 


If  you  need  any  more  information  about  this,  please  let  me  know. 


Best  regards. 


J.  Michael  Polich 


S.  James  Press,  Using  the  PISE  Criterion  to  Measure  the  Effects  of 
Imbalance  in  the  Analysis  of  Covariance,  The  Rand  Corporation,  N-1890- 
MRAL,  1983. 


JMP:lh 


Enclosures: 


Tables  1-3 

Attachment  1,  Design  Option  1 
Attachment  2,  Design  Option  2 
Attachment  3,  List  of  ADIs 


Attachment  4,  Linkages  of  ADIs 


cc:  Capt.  Louise  Wilmot,  Office  of  Accession  Policy 
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Table  1 


ADVERTISING  MIX  TEST  CELLS 


Advertising 
(SMillion  per  Year)8 

Cell 

Program 

Joint 

Advertising 

Service- 

Specific 

Advertising 

Percent  of 
Nation  in 
Cell 

A 

Control 

(FY82  Advertising  Level) 

16 

60 

76 

B 

Reduced  Service  Advertising 

16 

17 

8 

C 

Reduced  Joint  Advertising 

4 

60 

8 

D 

Increased  Joint  Advertising 

40 

17 

8 

Expenditure  rate  necessary  for  a  nationwide  program.  During  the 
test,  this  rate  will  be  prorated  to  the  size  of  copulation  in  the  test 
cell. 


Pi 
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Tabl*  2 


BALANCING  VARIABLES 
Advertising  Mix. Test  Design 


Variable  Name 

Abbreviation 

Definition 

Source 

Si2e  of  Population 

Enlistment  Rates 

SIZE* 

Number  of  males  age  17-21 
in  the  area,  1982 

DMDC  estimate 
from  Census 
data 

Army 

ARMY 

Number  of  male  high-quality 

DMDC  data 

Navy 

NAVY 

enlistments  (high  school 

from  MEPCOM 

Air  Force 

AF 

diploma  graduates  and 

ARS  system 

Marine  Corps 

MC 

seniors  with  AFQT  scores 

DoD 

DOD 

of  SO  or  above),  calendar 
year  1982,  as  a  percent 
of  1982  male  population 
age  17-21 

Nonwhite  Population 

NONV 

Black  and  Hispanic  males 
age  17-21,  as  a  percent  of 
total  male  population  age 
17-21,  1982 

Census 

Unemployment  Rate 

UNEM 

Unemployment  rate  in 
manufacturing,  all  ages, 
calendar  1982 

Bureau  of 
Labor 

Statistics, 

county  file 

Propensity  to  Enlist 

PROP 

Percent  of  area  survey 
respondents  who  reported 
positive  propensity  to 
enlist  ("definite"  or 
"probable"  intention) 

1982  Youth 
Attitude 
Tracking 
Study,  MEPS 
level  data 

Geographic  Region 

Percent  of  1982  area 

Census 

Northeast 

NE 

population  (males  17-21) 

Vest 

V 

living  in  each  major 

South 

S 

geographic  region 

Market  Size 

TOP  12 

Percent  of  area  1982  popu¬ 
lation  (males  17-21)  living 
in  one  of  the  12  most  popu¬ 
lous  ADIs  (which  make  up 
one-third  of  the  U.S. 
population) 

Census 

Cells  are  not  actually  balanced  on  this  variable.  It  is  used  to  define 
the  else  of  the  test  cell  and  as  an  eleaent  in  computation  of  the  other 
variables. 
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DCSICN  OPTION  2 

CCU  A  ($16M/$60M)  (cont. 
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OtjICN  OPT IOH  2 

CILL  A  ( St6M/$60M)  (cont. 


Liat 

of  ADI 1  a 

by  Size  of 

Market 

vX 

mn 

1  ADI  MAME  PRODUCTION 

CODE  MALES 

CUMULATIVE 

CELL*  A 

NUMBER 

17-21 

PERCENT 

PERCENT 

1 

NEW  YORK 

009 

680640 

7.039 

7.039 

2 

LOS  ANGELES 

013 

494032 

5.109 

12.148 

;< 

3 

CHICAGO 

051 

362063 

3.744 

15.892 

•  y* 

•V- 

4 

PHILADELPHIA 

on 

271260 

2.805 

18.698 

c  y£ 

5 

SAN  FRANCISCO 

065 

238079 

2.462 

21.160 

A-l 

6 

BOSTON 

003 

217891 

2.253 

23.414 

7 

DETROIT 

057 

212523 

2.198 

25.612 

8  J 

8 

DALLAS -FT.  NORTH 

109 

174125 

1.801 

27.413 

B  '< 

9 

WASHINGTON,  DC 

019 

170069 

1.759 

29.172 

> 

10 

HOUSTON 

201 

166770 

1.725 

30.897 

A-l  > 

11 

CLEVELAND 

035 

159423 

1.649 

32.545 

D  > 

12 

MMNEAPOLIS-ST.  PAUL 

107 

140193 

1.450 

33.995 

D 

13 

SEATLE-TACOMA 

105 

133033 

1.376 

35.371 

C  < 

14 

ATLANTA 

197 

131883 

1.364 

36.735 

A-i 

15 

PITTSBURGH 

029 

131766 

1.363 

38.098 

> 

16 

ST.  LOUIS 

075 

114253 

1.182 

39.279 

•V 

17 

DENVER 

241 

113841 

1.177 

40.456 

18 

MIAMI 

127 

107483 

1.112 

41.568 

c  Ov 

19 

BALTIMORE 

021 

101695 

1.052 

42.620 

A 

20 

SACRAMENTO-STOCKTON 

067 

101676 

1.052 

43.672 

A-l  /• 

21 

INDIANAPOLIS 

083 

94252 

0.975 

44.646 

c 

22 

SAN  DIEGO 

015 

92778 

0.960 

45.606 

.*.■ 

23 

PHOENIX 

275 

91368 

0.945 

46.550 

24 

PORTLAND,  OR 

233 

90610 

0.937 

47.487 

B 

25 

HARTFORD-NEW  HAVEN 

025 

89287 

0.923 

48.410 

i 

26 

SALT  LAKE  CITY 

291 

82471 

0.853 

49.264 

V. 

\ 

27 

CINCINNATI 

093 

81941 

0.847 

50.111 

A-l  X' 

28 

MILWAUKEE 

111 

80913 

0.837 

50.948 

a-i 

29 

TAMPA-ST.  PETERSBURG 

131 

79533 

0.823 

51.771 

s 

30 

KANSAS  CITY 

157 

78004 

0.807 

52.578 

A-l 

31 

NEW  ORLEANS 

245 

77310 

0.800 

53.377 

D  « 

32 

NASHVILLE 

181 

74398 

0.769 

54.146 

33 

RALEIGH-DURHAM 

351 

71777 

0.742 

54.888 

A-l  '••j 

34 

GRAND  RAP IDS -KALAMAZ 00- 

BATTLE  CREEK 

059 

71045 

0.735 

55.624 

.  « 

35 

COLUMBUS,  OB 

121 

70224 

0.726 

56.349 

C 

36 

CHARLOTTE 

279 

69437 

0.718 

57.068 

A-l  • 

37 

MEMPHIS 

179 

68218 

0.706 

57.774 

d 

18 

SAN  ANTONIO 

271 

65771 

0.680 

58.453 

d  y 

39 

BUFFALO 

135 

64668 

0.669 

59.122 

A-l 

44 

LOUISVILLE 

209 

62476 

0.646 

59.768 

►3 

41 

GRXXNVILLE-SPARTANBURG- 

0 

AMIVILLZ 

213 

61822 

0.639 

60.407 

4 

42 

PROVIDENCE-NEW  BEDFORD 

047 

61694 

0.638 

61.046 

a 

43 

OKLAHOMA  CITY 

263 

61596 

0.637 

61.683 

.-j 
*  \ 

44 

NORPOLK-PORTSMOUTH- 

s 

KNPORT  NEWS-HAMP 

283 

61488 

0.636 

62.319 

$ 

45 

dARLtSTON-IUNTI  NGTON 

257 

60390 

0.625 

62.944 

<4 

U 

IWn  Hrlfti  *»t  labeled  fall  into  the  larger  control  cell  A  (White-76Z  of 

mmry).  W 

*M  »Mree—t :  Aj  -  Yello  Control  Cell  W 


•  -  line 
C  •  Creen 
B  ■  Bed 
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:s 
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NLKBER 

ADI  MAW  PROMOTION 

CODE  MALES 

CUMULATIVE 

CELL 

Hum  EB 

17-21 

PERCENT 

PERCENT 

46 

ORLANDO-  DAYTONA  1IACV 

329 

59162 

0.612 

63.555 

>47 

flint-sag inav-bay  cm 

063 

57052 

0.590 

44.145 

<*8 

DAYTON 

095 

55861 

0.576 

44.723 

49 

BIRMINGHAM 

221 

54746 

0.544 

65.289 

*>0 

1AREISB0HC-Y0RK -LANCASTER- 

LEBANON 

043 

51290 

0.530 

65.619 

B 

M 

M061LI- PENSACOLA 

3S3 

51190 

0.529 

44.349 

:2 

Richmond 

215 

51142 

0.529 

44.676 

53 

GREENS BOHO-V I NSTON  SALEM- 

HIGH  POINT 

281 

49859 

0.514 

67.394 

54 

SHREVEPORT -TEXARKANA 

321 

46*67 

0.504 

47.900 

,5 

ms  HO 

071 

46619 

0.505 

46.404 

56 

TOLEDO 

055 

47654 

0.495 

46.699 

s  7 

ALBANY -SCKENECTADY-TROY 

149 

47824 

0.495 

49.394 

58 

LITTLE  KOCE 

319 

47744 

0.494 

69.866 

59 

WILKES  BAEEE-SCIAHTOH 

143 

47577 

0.492 

70.360 

A- 

50 

ALBUQUERQUE 

347 

44399 

0.460 

70.640 

M 

KNOXVILLE 

215 

45578 

0.471 

71.331 

62 

vicnita-nutchinson 

307 

45202 

0.447 

71.796 

63 

TULSA 

249 

44195 

0.457 

72.255 

C 

44 

JACKSONVILLE 

335 

43618 

0.451 

72.704 

B 

45 

KQAMKE-LTHCBBUBC 

345 

42355 

0.438 

73.145 

46 

GKEED  SAT 

315 

42104 

0.435 

73.560 

67 

DES  MOINES 

303 

40347 

0.417 

73.997 

48 

CEDAE  IAPIDS-WATEELOO 

173 

39318 

0.407 

74.404 

69 

OHAIA 

301 

38232 

0.395 

74.799 

70 

STBACUSE 

141 

36107 

0.394 

75.193 

71 

KOCBESTEK,  HT 

139 

37307 

0.386 

75.579 

72 

SPRINGFIELD- DECATUB- 

CHAMPAICN 

077 

37278 

0.386 

75.965 

’3 

AUSTIN,  TX 

203 

37015 

0.383 

76.346 

•4 

SPOKANE 

337 

36772 

0.380 

76.728 

7  5 

DAVENPORT -BOCK  ISLAND- 

MOLINE 

177 

36515 

0.378 

77.106 

A- 

?6 

BATON  BOUCE 

249 

36328 

0.376 

77.482 

77 

LEXINGTON 

211 

35777 

0.370 

77.851 

A- 

78 

PADUCAH -CAPE  GIRARDEAU- 

HAKXISBUBG  G 

187 

35235 

0.364 

78.215 

A- 

T 

BBISTON-KINGSPORT-JOHNSON- 

CITT 

217 

34715 

0.359 

78.575 

80 

PORTLAND- POLAND  SPRING 

123 

34329 

0.355 

78.930 

81 

JORNSTOVN-ALTOONA 

033 

32864 

0.340 

79.270 

82 

TUCSON 

277 

32725 

0.338 

79.608 

A- 

83 

COLUMBIA,  SC 

361 

32630 

0.337 

79.945 

84 

CHATANOOGA 

199 

32261 

0.334 

80.279 

85 

SOUTH  BEND-ELKHART 

053 

32148 

0.332 

80.611 

86 

SPRINGFIELD,  MO 

427 

31557 

0.326 

80.938 

B 

87 

VACO-TEMPLE 

205 

31317 

0.324 

81.262 

B 

88 

JACKSON,  MISS. 

373 

31105 

0.322 

81.584 

89 

LANSING 

061 

30957 

0.320 

81.904 

90 

VEST  PALM  BEACH 

129 

30513 

0.316 

82.220 

C 

91 

EL  PASO 

371 

30078 

0.311 

82.530 

B 

B-2? 
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401  HAW  PROMOTION  COM 

H*IM 

CUMULATIVE 

CELL 

■NIMB 

17-21 

PERCENT 

PERCENT 

$2 

L I NCOLN-RASTINCS -KEARNEY 

331 

28931 

0.299 

12.829 

93 

PALM  SPRINGS 

577 

28392 

0.296 

83.126 

94 

HUNTS  V  I LLE-DECATUR- 

PLORENCE 

185 

28333 

0.295 

13.420 

A- 1 

95 

COLUMBUS ,  GA 

409 

28419 

0.294 

83.715 

9* 

MADISON 

113 

20133 

0.291 

84.005 

97 

CREENVILLE-NEV  SEE N- 

WAS 1 INC TON 

353 

27623 

0.284 

•4.292 

D 

98 

SPBINCFIELD.  MASS. 

045 

27184 

0.281 

84.572 

99 

TOONC8TONN 

031 

26797 

0.277 

•4.849 

D 

100 

COLORADO  SPRINGS -PUEBLO 

243 

26767 

0.277 

85.127 

101 

rr.  WAYNE 

091 

26743 

0.277 

•5.403 

102 

PEON  LA 

175 

26403 

0.273 

•5.676 

103 

PARGO 

393 

23760 

0.266 

15.942 

104 

EVANSVILLE 

207 

25337 

0.264 

•6.207 

D 

105 

SAVANNAS 

425 

23476 

0.263 

•6.460 

106 

SORL INC TON -PLATTSBURGH 

151 

23931 

0.248 

•6.718 

107 

LAPATETTE,  LA 

233 

23913 

0.247 

86.965 

108 

SANTA  BARBARA-SANTA  MAI LA¬ 

BAN  LUIS  OBI 

017 

23881 

0.247 

•7.212 

109 

MCALLEN- BROWNSVILLE 

435 

23710 

0.245 

87.457 

110 

LAS  VEGAS 

435 

23683 

0.245 

•7.702 

111 

CNARLESTON,  SC 

423 

23638 

0.245 

•7.947 

112 

ROCEPONO 

119 

23484 

0.243 

•8.190 

113 

AUGUSTA 

421 

23178 

0.240 

•8.430 

114 

SIOUX  PALLS - HITCNELL 

389 

23172 

0.240 

18.670 

A- 1 

115 

SAL I HAS -MONTEREY 

069 

23116 

0.239 

68.908 

A- 1 

116 

CORPUS  CIRISTI 

433 

22400 

0.232 

89.140 

A- 1 

117 

TALLAHASSEE 

413 

22319 

0.231 

89.371 

118 

MONROE- EL  DORADO 

327 

21243 

0.220 

89.591 

119 

EUGENE 

235 

20980 

0.217 

89.807 

B 

120 

LUBBOCK 

437 

20693 

0.214 

90.021 

121 

WICHITA  PALLS-LAWTON 

405 

19946 

0.206 

90.227 

122 

TOPEKA 

313 

19866 

0.205 

90.433 

A- 1 

123 

LA  CKOSSE-EAU  CLAIRE 

117 

19699 

0.204 

90.637 

A- 1 

124 

AMARILLO 

403 

19686 

0.204 

90.841 

125 

WILMINGTON 

355 

19240 

0.199 

91.040 

126 

YARIMA 

339 

19182 

0.198 

91.237 

A-l 

127 

MONTGOMERY 

411 

19056 

0.197 

91.435 

128 

BEAUMONT-PORT  ARTHUR 

247 

18922 

0.196 

91.631 

129 

DULUTH -SUPER I OR 

381 

18888 

0.195 

91.826 

130 

BOISE 

445 

17893 

0.185 

92.011 

131 

WAUSAU -RBI MELANDE R 

115 

17368 

0.180 

92.191 

A-l 

132 

BAKERSFIELD 

073 

17294 

0.179 

92.370 

133 

COLUMBUS-TUPELO 

448 

17270 

0.179 

92.548 

134 

JOPLIN-PITTSBURG 

429 

17216 

0.178 

92.726 

A-l 

135 

WHEELING-STEUBENVILLE 

103 

17036 

0.176 

92.902 

136 

MACON 

219 

16763 

0.173 

93.075 

A-i 

137 

TERRE  HAUTE 

087 

16565 

0.171 

93.247 

138 

MINOT-BISMARCK -DICKINSON 

462 

16340 

0.171 

93.418 

139 

ERIE 

147 

16452 

0.170 

93.588 

140 

BLUEFIELD-BECKLEY-OAX  HILL 

347 

16275 

0.168 

93.756 

141 

SIOUX  CITY 

391 

16238 

0.168 

93.925 

B-28 
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NUMBER 

ADI  NAME  PRODUCTION  CODE 

MALES 

CUMULATIVE 

CELL 

NUMBER 

17-21 

PERCENT 

PERCENT 

142 

COLUMBIA- JEFFERS ON  CITY 

229 

16221 

0.168 

94.092 

143 

BINGHAMTON 

145 

15934 

0.165 

94.257 

144 

RENO 

459 

15110 

0.156 

94.413 

143 

MISSOULA-BUTTE 

342 

15086 

0.156 

94.569 

146 

CHICO- REDDING 

089 

14958 

0.155 

94.724 

A-l 

147 

TRAVERSE  CITY-CADILLAC 

451 

14863 

0.154 

94.878 

148 

FT.  SMITH 

325 

14861 

0.154 

95.032 

C 

149 

ODESSA-MIDLAND 

439 

14261 

0.147 

95.178 

150 

ROCHESTER-MASON  CITY- 

AUSTIN 

165 

14184 

0.147 

95.326 

151 

UTICA 

155 

14168 

0.147 

95.473 

132 

IDAHO  FALLS-POCATELLO 

295 

13747 

0.142 

95.614 

A-l 

153 

BANGOR 

357 

13647 

0.141 

95.755 

154 

ALBANY,  GA 

419 

13560 

0.140 

95.895 

155 

ALEXANDRIA,  LA 

255 

13384 

0.138 

96.034 

156 

QUINCT-HANNIBAL 

227 

13085 

0.135 

96.169 

157 

GAINESVILLE 

621 

13058 

0.135 

96.304 

B 

158 

FLORENCE,  SC 

359 

13045 

0.135 

96.439 

159 

RAPID  CITY 

469 

12901 

0.133 

96.572 

160 

FT.  MYERS-NAFLES 

133 

12559 

0.130 

96.703 

161 

MEDFORD 

237 

12129 

0.125 

96.828 

162 

AB ILENE-S VEETVATER 

441 

11445 

0.118 

96.946 

B 

163 

DOTHAN 

415 

10530 

0.109 

97.055 

164 

LAUREL-HATTIESBURG 

379 

10312 

0.107 

97.162 

A-l 

165 

TYLER 

323 

10235 

0.196 

97.268 

B 

166 

BILOXI-GULFPORT- PASCAGOULA 

363 

9829 

0.102 

97.370 

D 

167 

LAKE  CHARLES 

251 

9527 

0.099 

97.459 

168 

PANAMA  CITY 

417 

9510 

0.098 

97.566 

169 

BILLING-HARDIN 

457 

9207 

0.095 

97.661 

170 

ELMIRA 

140 

8758 

0.091 

97.753 

171 

ALEXANDRIA ,  MINN. 

395 

8436 

0.087 

97.839 

D 

172 

WATERTOHN-CARTHAGE 

153 

8396 

0.087 

97.926 

173 

GREENUOOD-GREENVILLE 

375 

8246 

0.085 

98.011 

174 

SALISBURY 

023 

8212 

0.085 

98.097 

175 

EL  CENTRO- YUMA 

039 

7981 

0.083 

98.179 

176 

CLARKSBURG-WESTON 

261 

7827 

0.081 

98.260 

177 

GREAT  FALLS 

299 

7715 

0.080 

98.340 

178 

TUSCALOOSA 

231 

7638 

0.079 

98.419 

179 

CASPER-RIVERTON 

471 

7591 

0.079 

98.498 

D 

180 

JONESBORO 

431 

7275 

0.075 

98.572 

D 

181 

LAFAYETTE,  IND. 

085 

7023 

0.073 

98.645 

C 

182 

ARDMORE-ADA 

265 

6873 

0.071 

98.716 

183 

EUREKA 

467 

6866 

0.071 

98.787 

A-l 

184 

MARQUETTE 

317 

6507 

0.067 

98.854 

185 

MERIDIAN 

377 

6438 

0.067 

98.922 

186 

ROSWELL 

369 

6373 

0.066 

98.987 

187 

CHEYENNE 

465 

6158 

0.064 

99.051 

D 

188 

BOWLING  GREEN 

195 

5687 

0.059 

99.110 

189 

SARASOTA 

645 

5593 

0.058 

99.168 

190 

ANNISTON 

603 

5579 

0.058 

99.225 

191 

HARRISONBURG 

287 

5524 

0.057 

99.282 

192 

MANKATO 

449 

5257 

0.054 

99.336 

D 

s$ 
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RMW 

ADI  KAMI 

PftOOOCTION  CODE 
NDMSEK 

MALES 

17-21 

PEtCENT 

CUMULATIVE 

PEtCENT 

CELL* 

193 

(HARD  JUNCTION 

473 

5070 

0.052 

99.369 

D 

194 

8T.  J08EPI 

159 

4903 

0.051 

99.440 

A-l 

195 

LA1KDO 

273 

4806 

0.050 

99.490 

D 

196 

LIKA 

101 

4600 

0.048 

99.537 

197 

JACKS 0H(  TUI. 

183 

4581 

0.047 

99.564 

D 

198 

SAN  ANGKLO 

443 

.4251 

0.044 

99.626 

8 

199 

PA1KEXSBUXC 

259 

3807 

0.039 

99.667 

200 

PtESQUI  ISLE 

161 

3646 

0.38 

99.706 

201 

OTTUMMAHC  IKES  VILLE 

305 

3577 

0.037 

99.742 

202 

FASMINGTOM 

649 

3498 

0.036 

99.778 

203 

TWIN  FALLS 

293 

3354 

0.035 

99.814 

204 

VICTORIA 

513 

3210 

0.033 

99.846 

A-l 

205 

ZANESVILLE 

125 

3112 

0.032 

99.878 

C 

206 

SELMA 

225 

3054 

0.032 

99.911 

207 

SIND 

591 

2471 

0.026 

99.936 

B 

206 

HELENA 

297 

1765 

0.018 

99.954 

209 

NOKTN  PLATTE 

385 

1650 

0.017 

99.971 

210 

ALPENA 

627 

1552 

0.016 

99.966 

211 

MILES  CITT-CLENDIVE 

653 

1245 

0.013 

100.000 

B-30 
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ADI  LINXAQES  FOR  ADVERTISING  NIX  TEST 
(MINIMIZING  IS*.  SPILL-IN) 


LINKAGE 

GROUP 


RANCOR  HE 
PRESQUE  ISLE  ME 


ROSTON  NA 

PROVIDENCE  RI -NEW  REDFORD  NA 
PORTLAND -POLAND  SPRING  Nt 


NEW  YORK  NY 
HARTFORD -NEW  HAVEN  CT 
SPRINGFIELD  NA 


SYRACUSE  NY 

UTICA  NY 
WATERTOWN  NY 


SINGHAMTON  NY 
ELMIRA  NY 


SALT1NORE  HD 
SALISBURY  MD 


WASHINGTON  DC 
HARRISONBURG  VA 
RICHMOND  VA 


SLUEFIELD-SECKLEY-OAK  HILL  WV 
CHAR  LE  STON - HUNT I NGTON  WV 

PARKERSBURG  WV 


PITTSBURGH  PA 
JOHNSTOWN -ALTOONA  PA 
WHEELING  WV- STEUBENVILLE  OH 
C LARKS IURG -WESTON  WV 


CLEVELAND  OH 
YOUNGSTOWN  OH 


COLUMBUS  OH 
ZANESVILLE  OH 


TOLEDO  OH 
LIMA  OH 


FLINT-SAGINAV-BAY  CITY  MI 
APLENA  MI 

TRAVERSE  CITY -CADILLAC  MI 


*  -  '  w*  Vs  A.'*  V**  .V  A  ' .  A  J, 


LINKAGE 

QSOUP 


ADI 

NAME 


14 


I) 


1* 


1? 


It 


19 


20 


21 


22 


23 


24 


25 


26 


27 


INDIANAPOLIS  IN 
LATATtm  IN 


GREEN  tAY  VI 

NAJKjutrrt  hi 


MINNEAPOLIS -ST  PAUL  NN 
NANKATO  NN 
ALEXANDRIA  NN 


OTTUNVA  lA-klRKSVlLU  NO 
QUINCY  IL-MANNIBAL  HO 


OHARA  NC 

LINCOLN- HAITI NGS  NL 
Norm  PLATTE  NE 


KANSAS  CITY  NO 
ST  JOSEPH  NO 
TOPEKA  KS 


NENPN1S  TN 
JONESBORO  AS 
JACKSON  TYi 


NASHVILLE  TN 
BOWLING  GREEN  KY 


ATLANTA  GA 
MACON  GA 


COLUMBIA  SC 
FLORENCE  SC 


BIRMINGHAM  AL 
ANNISTON  AL 
TUSCALOOSA  AL 


JACKSON  NS 

GREENWOOD-GREENY! LLE  MS 


JACKSONVILLE  FL 
GAINESVILLE  FL 


TAMPA -ST  PETERSBURG  FL 
SARASOTA  FL 


MIAMI  FL 

VEST  PALM  BEACH  FL 


IINXASC 

GROUP 


MOV7 GONER Y  AL 
SLuA  AL 
PANAMA  CITY  FL 
DOTHAN  AL 

TALLAHASSEE  FI 

ALBANY  GA 

NEW  ORLEANS  LA 
BILOXI -GULFPORT  MS 

BLA'  MONT  TX 
Ui  AUTTL  LA 
I-AkF.  CHARLES  LA 
ALEXANDRIA  LA 

DALLAS  TX 
TYLER  TX 

ABILLNi  -SWEETWATER  T\ 
SAN  ANGELO  TX 
W  AUj-  TEMPLE  TX 

Ok LAH(>MA  CITY  "► 

AKD'I-  Ri  k 

ULNA  OK 
FT  SMITH  AR 

SAN  ANTONIO  TX 
VICTORIA  TX 
LAREDO  TX 

LUBBOCK  TX 
ROSWELL  N.M 

ALBUQUERQUE  NM 
FARMINGTON  N’M 

DENVER  CO 
GRAND  JUNCTION  CO 
CHEYENNE  VY 
CASPER-RIVERTON  VY 

SALT  LAKE  CITY  UT 
TWIN  FALLS  ID 
HELENA  MT 

MILES  CITY-GLENDIVE  MT 
MISSOULA -BUTTE  MT 
BOISE  ID 
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APPENDIX  C 


THE  FY  1984  ADVERTISING  MIX  TEST 


applied  Research  Center 
The  Wharton  School 
University  of  Pennsylvania 
Philadelphia.  Pennsylvania  19104 


PREFACE 


This  report  has  been  prepared  under  Office  of 
Naval  Research  Contract  N00014-83-C-0663.  It 
is  die  final  report  for  Task  1  described  in  the 
Management  Plan  prepared  for  the  FY  1984 
Advertising  Mix  Test  and  constitutes  Subtask  1.6 
of  that  Plan. 

The  information  in  this  report  was  the  most 
current  available  as  of  15  June  1984.  Source 
documents  include: 

JRAP  Fact  book  dated  9  April  1984  as  updated 
by  telephone 

Army  Factbook  dated  7  November  1984 


Navy  Factbook  dated  12  June  1984 
USAF  Factbook  dated  November  1983 
USMC  Factbook  transmitted  31  May  1984 
with  addendum  per  J.  Walker  Thompson 
dated  S  June  1984. 

The  report  was  prepared  by  Vincent  P. 
Carroll,  principal  investigator  for  the  FY  1984 
Advertising  Mix  Test,  and  Associate  Director  of 
the  Wharton  Applied  Research  Center  (W  ARC)  at 
the  University  of  Pennsylvania  and  by  Judith 
Mauer,  Project  Manager. 


Ul 


rn  H  wvwj  »v»,jr.'nTi[ 


CONTENTS 


I 


K. 


k 

k 

«, 

k 

K 

k 

! 


BACKGROUND 

REVIEW  OF  ADVERTISING  AND  TRANSLATION  PLANS 
Experimental  Designs 
Advertising  Plans 
Transition  Plans 


SUMMARY  OF  PLANNED  MEDIA  ACTIVITY 
Ninons  I  Advertising  Media  Expenditures 

;  a i  NaooaaJ  Advertising  Media  Schedules 


FIGURES 

I 


Proportional  AUocaoon  of  Simulated  Budgets  to  Working 
and  Non- Working  Media  Expenditures  Category 

2  FY  1984  Mmfca  Schedules  By  Treatment  Condition 


TABLES 


1  huoal  Design  of  the  FY  1984  Advertising  Mix  Test 

2  Folded  Design  of  the  FY  1984  Advertising  Mix  Test 

3  Allocation  of  Simulated  Advertising  Budgets  By  Test  Cell 
and  Expenditure  Category 

4  Working  Media  Expenditures  Planned  For  Each  Test  Cell 

5  Working  Media  Expenditures  Implied  by  Advertising  Plans  vs. 
Working  Media  Expenditures  Observed  in  Translation  Plans 

6  Planned  Working  Media  Expenditures  Per  17-21 -Year-Old  Male 

7  Planned  Media  Expenditures  for  National  Advertising 

8  Planned  Media  Expenditures  for  National  Advertising 


EASE 

1 

1 

1 

3 

6 

9 

9 

11 


i 

4 


3 

r 

f 

f 

t 

i 

i 


l 


5 

12 


2 

3 

4 

6 

7 

9 

10 

11 


775  MO  (DEPARTMENT  OF  DEFENSE)  ADVERTISING  NIX  TEST 
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THE  FY 1984  ADVERTISING  MIX  TEST 
REVIEW  OF  PLANNED  ADVERTISING  EXPENDITURES 


BACKGROUND 

The  primary  purpose  of  the  FY  1984 
Advertising  Mix  Test  is  to  help  the  Department  of 
Defense  determine  the  optimum  budget  level  for 
DoD  recruiting  advertising  and  the  best  mix  of 
Joint  and  Service-specific  advertising.  Directed 
by  the  Secretary  of  Defense,  the  test  varies  the 
level  and  mix  of  active,  enlisted,  non-prior- 
service  (NPS)  advertising  expenditures  in  four 
matched  groups  of  markets.  The  experimental 
design  for  the  test,  established  in  an  8  July  1983 
memorandum  from  the  Assistant  Secretary  of 
Defense  for  Manpower,  Installations  and 
Logistics,  was  subsequently  modified  in  response 
to  budgetary  constraints.  To  prepare  for  the  test, 
the  advertising  directors  for  the  Joint  Recruiting 
Advertising  Program  (JRAP)  and  the  Services 
submitted  two  planning  documents:  an  advertising 
plan  allocating  the  national  budget  levels  being 
simulated  in  each  treatment  condition  between 
working  media  and  non-working  media  expendi¬ 
tures;  and  a  translation  plan  for  simulating  the 
hypothetical  national  budget  for  each  treatment 
condition  in  its  associated  test  cell. 

As  the  prime  contractor,  Wharton  Applied 
Research  Center  (WARQ  is  responsible  for 


providing  a  series  of  reports  documenting  the 
planning,  implementation,  data  collection,  and 
analysis  phases  of  the  year-long  test  This  report 
compares  advertising  expenditures  planned  by 
JRAP  and  the  Services  with  the  modified  experi¬ 
mental  design  and  summarizes  planned  media 
activity. 

REVIEW  OF  ADVERTISING  AND 
TRANSLATION  PLANS 

Experimental  Designs 

Table  1  presents  the  budget  levels  envisioned 
in  the  original  concept  design.  The  control  cell 
(White)  is  based  on  actual  FY  1982  advertising 
expenditures  for  JRAP  and  the  Services.  The 
three  experimental  conditions  are  designed  to  test 
a  wide  range  of  total  DoD  recruiting  advertising 
budget  levels  and  different  budget  allocations  to 
JRAP  and  Service-specific  advertising.  Appor¬ 
tionment  of  the  Service-specific  advertising 
budget  to  individual  Services  is  fixed  according  to 
historical  (FY  1979-81)  advertising  budget  shares 
and  is,  therefore,  not  a  treatment  variable. 


Cell 


Coverage 
of  Youth 
Market 


Advertiser 


Simulated 
National 
Budgets  ($M) 


Cell 
Design 
Buget  ($M) 


•Totals  may  not  add  due  to  rounding 


As  a  result  of  reductions  in  funds  available  to 
JRAP  and  Navy,  the  original  concept  design  was 
modified.  The  Navy  reduction  of  $2.1  million 
was  distributed  proportionally  across  all  cells. 
JRAP  maintained  the  design  level  of  spending  in 
all  experimental  conditions  and  in  a  matched 
subset  of  the  control  cell  markets  consisting  of 


16%  of  the  youth  population  (Cell  White  Al).  The 
$2.3  million  underfunding  of  JRAP's  increased 
test  costs  was  entirely  absorbed  in  the  remainder 
of  the  control  cell  (Cell  White  A).  These  funding 
adjustments  preserved  the  design  characteristics 
of  the  test  Table  2  presents  the  modified  test 
design. 


Tabic  2 

Fielded  Design  of  the  FY 1984  Advertising  Mia  Test 


Cell 

Coverage 
of  Youth 

Market 

Advertiser 

Simulated 

National 

Budgets  ($M) 

Ceil 
Design 
Buget  ($M) 

JRAP 

12.0 

7.2 

White  A 

60% 

Services 

65.1 

39.1 

Total* 

77.1 

46.3 

JRAP 

15.8 

2.5 

White  A1 

16% 

Services 

65.1 

10.4 

Total* 

80.9 

12.9 

JRAP 

15.8 

1.3 

Blue 

8% 

Services 

14.6 

1.2 

Total* 

30.4 

2.4 

JRAP 

4.0 

0.3 

Green 

8% 

Services 

65.1 

5.2 

‘ 

Total* 

69.1 

5.5 

JRAP 

40.0 

3.2 

Red 

8% 

Services 

14.6 

1.2 

Total* 

54.6 

4.4 

♦Totals  may  not  add  due  to  rounding 


Advertising  Plans 

Each  of  the  military  recruiting  advertisers 
prepared  advertising  plans  for  each  of  the  four 
treatment  conditions.  These  plans  established 
levels  of  spending  for  local  advertising,  national 
advertising  and  production,  labor  and  administra¬ 


tive  categories.  In  keeping  with  current  and 
historical  practice,  the  proportion  of  the  total 
budget  allocated  to  each  of  these  categories  was 
not  constrained  by  the  test  design.  Table  3 
summarizes  these  allocations  for  the  treatment 
conditions. 


s' 


3 


> 


Table  3 


I 


Allocation  of  Simulated  Advertising  Budgets  By  Test  Cell 
and  Expenditure  Category 


Figure  1  depicts  the  proportional  allocation  of 
the  total  DoD  advertising  budget  to  working  and 
non-working  media  categories.  Nearly  identical 
proportional  allocations  are  evident  for  Cells 
White  and  Blue.  Cell  Green  has  a  higher 
aggregate  proportional  allocation  to  the  local 
advertising  and  production  categories  while  Cell 
Red  has  a  proportionately  higher  allocation  for 
national  advertising.  These  allocation  patterns  are 


consistent  with  the  strengths  proposed  for  Service- 
specific  and  JRAP  advertising,  respectively.  That 
is,  the  reliance  on  local  advertising  in  Cell  Green 
provides  for  a  stronger  linking  mechanism 
between  advertising  and  the  recruiting  force  while 
Cell  Red  reflects  production  efficiencies  resulting 
from  lower  production  costs.  The  next  section 
reviews  plans  for  implementing  these  advertising 
plans  in  the  test  cells. 
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Figure  1.  Proportional  Allocation  of  Simulated  Budget8  to  Working 

and  Non-Working  Media  Expenditure  Categories 


Translation  Plans 

Having  established  plans  for  distributing 
working  and  non-working  media  expenditures 
appropriate  to  each  hypothetical  national  budget. 


the  advertisers  prepared  translation  plans  for 
implementing  their  decisions  in  die  test  cells. 
Table  4  summarizes  waking  media  expenditures 
planned  by  JRAP  and  the  Services. 


Cell 


Tabled 

Working  Media  Expenditures  Planned  for  Each  Test  Cell 


Males 
17-21  Yean 
(000) 


Total  Working  Local  National 

Media  Budget  Advertising  Advertising 

($M)  ($M)  ($M) 


Advertiser 


Working  Media  Correspondence  Be¬ 
tween  Advertising  and  Translation 
Plans 

Table  5  compares  the  total  working  media 
expenditures  planned  for  each  cell  with  those 
implied  by  the  advertising  plans.  An  examination 
of  this  table  reveals  some  planned  expenditures 
significantly  in  excess  of  those  implied  by  the 
advertising  plans.  For  example,  in  the  treatment 
condition  simulated  by  Cell  Red,  the  Services' 


advertising  plans  allocated  only  about  70%  of 
their  total  national  budgets  to  working  media. 
Applying  this  allocation  to  the  Services'  Cell  Red 
budgets  implies  working  media  expenditures  of 
$0,830  million  in  the  cell.  The  Services' 
translation  plans  show  working  media  allocations 
of  $1,024  million,  a  discrepancy  of  about  23%. 
Similar  discrepancies  are  observed  in  the  other 
cells  for  both  JRAP  and  the  Services.  These 
differences  occur  primarily  because  the  test  allows 


Table  5 

Working  Media  Expenditures  Implied  by  Advertising  Plans  vs. 
Working  Media  Expenditures  Observed  in  Translation  Plans 


Cell  Blue 

Cell  Green 

Cell  Red 

JRAP  Services 

JRAP  Services 

JRAP  Services 

Working  Media  Percentage 
from  Advertising  Plans 

89.2 

69.2 

77.5 

78.8 

93.3 

69.2 

Cell  Budgets  ($M) 

1.3 

1.2 

0.3 

5.2 

3.2 

1.2 

Implied  Cell  Working 

Media  Budget  ($M) 

1.160 

0.830 

0.233 

4.098 

2.984 

0.830 

Working  Media  Budget 
from  Translation  Plan  ($M) 

1.125 

0.978 

0.303 

4.389 

2.985 

1.024 

%  Deviation 

-3.0 

17.8 

30.0 

7.1 

0.0 

23.4 

use  of  some  national  media  (e.g.,  national  maga¬ 
zines)  which  cannot  effectively  be  blanked  out  in 
the  test  markets.  Although  some  overdelivery  in 
the  test  cells  was  expected  to  result  from  these 
media,  prohibiting  their  use  could  have  severely 
biased  the  test  Planned  overdeliveries  of  JRAP 
advertising  in  Cell  Green  and  of  Service 
advertising  in  Cells  Blue  and  Red  appear  to  result 
from  use  of  these  media. 

Correspondence  of  Per  Capita  Expen* 

ditures  Across  Cells 

Despite  this  variance  between  the  translation 
plans  and  advertising  plans,  the  cross-cell 
comparisons  of  per  capita  expenditures  in  Table  6 
indicate  that  the  total  design  characteristics  have 
been  maintained  in  the  aggregate.  As  total  cell 
budgets  decline  (from  Cell  White  to  Cell  Blue), 
per  capita  media  expenditures  also  decline.  More¬ 
over,  the  relationships  between  JRAP  or  Services 
cell  budgets  and  per  capita  media  expenditures 
across  cells  are  generally  maintained.  For 
example,  in  Cells  Green  and  Red,  JRAP  is 
simulating  national  budgets  of  $4  million  and  $40 
million,  respectively.  The  per  capita  media 
expenditures  by  JRAP  in  Cells  Green  and  Red, 
$0.39  and  $3.91,  respectively,  generally  reflect 
the  same  1:10  ratio.  Similarly,  in  cells  with  the 
same  total  Service  budget  (e.g.,  Cells  Blue  and 
Red),  the  Service  per  capita  expenditure  levels  are 


identical.  Other  cross-cell  comparisons  are 
generally  consistent  with  these  examples. 

Correspondence  of  Per  Capita  Expen¬ 
ditures  Within  Cells 

Within -cell  comparisons  of  per  capita 
expenditures  for  the  JRAP  and  Services'  plans 
also  conform  to  the  experimental  design.  For 
example,  in  Cell  Red  where  JRAP  and  the 
Services  are  simulating  national  budgets  of  $40 
million  and  $14.6  million,  respectively,  their  per 
capita  media  expenditure  of  $3.91  and  $1.34 
approximate  the  same  ratio.  Within-cell  compari¬ 
sons  for  the  other  treatment  conditions  exhibit  this 
same  general  correspondence. 

Conclusions 

The  apportionments  of  simulated  national 
budgets  between  working  and  non-working 
media  categories  in  the  advertising  plans 
submitted  by  JRAP  and  the  Services  appear  to  be 
reasonable  and  realistic. 

Although  the  translation  plans  do  not  exactly 
replicate  the  proportional  allocations  to  working 
and  non-working  media  categories  observed  in 
the  advertising  plans,  the  structure  of  the  planned 
per  capita  media  expenditures  is  highly  consistent 
with  the  fielded  experimental  design. 


Table  6 

Planned  Working  Media  Expenditures  Per  17-21-Year-Old  Male 


f  »Vi  WWW  7^  rwLnm 


CeU 

Advertiser 

Planned 

Per  Capita 

Expenditures  ($) 

JRAP 

1.05 

White  A 

Services 

5.40 

Total* 

6.46 

JRAP 

1.48 

White  A1 

Services 

5.51 

Total* 

6.99 

JRAP 

1.55 

Blue 

Services 

1.34 

Total* 

2.89 

JRAP 

0.39 

Green 

Services 

5.65 

Total* 

6.04 

JRAP 

3.91 

Red 

Services 

1.34 

Total* 

5.25 

•Totals  may  not  add  due  to  rounding 

SUMMARY  OF  PLANNED  NATIONAL 
MEDIA  ACTIVITY 

The  second  purpose  of  this  report  is  to 
summarize  national  media  activity  planned  by 
JRAP  and  the  Services. 


National  Advertising  Media  Expenditures 
Table  7  presents  a  summary  of  planned 
national  advertising  by  media  category,  and  Table 
8  expresses  those  planned  expenditures  as  per¬ 
cents  of  advertisers'  cell  budgets. 
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Table  7 


f 

\ 


Planned  Media  Expenditures  for  National  Advertising 
(Millions  of  Dollars) 


Cell 

Advertiser 

TV 

Radio 

Mag/ 

Print 

Direct 

MaU 

Outdoor 

Total* 

JRAP 

4.9 

0.0 

1.2 

6.2 

White  A 

Services 

12.9 

3.9 

6.8 

1.39 

0.14 

25.1 

Total* 

17.8 

3.9 

8.0 

1.47 

0.14 

31.3 

JRAP 

1.7 

0.0 

0.6 

0.02 

0.00 

2.3 

White  A1 

Services 

3.4 

1.0 

1.8 

0.37 

0.04 

6.6 

Total* 

5.1 

1.0 

2.4 

0.39 

0.04 

8.9 

JRAP 

0.3 

0.00 

1.1 

Blue 

Services 

0.5 

M'-’; 

0.01 

Total* 

0.8 

0.8 

0.12 

0.01 

1.7 

JRAP 

mm 

0.2 

0.01 

0.00 

Green 

Services 

1.8 

0.9 

0.19 

0.05 

3.5 

Total* 

1.9 

0.6 

1.1 

0.20 

0.05 

3.8 

JRAP 

2.6 

0.3 

0.02 

0.00 

Red 

Services 

0.02 

2.6 

0.12 

0.02 

3.6 

‘Totals  may  not  add  due  to  rounding 


The  data  in  Tables  7  and  8  clearly  reflect  that 
in  each  cell,  television  is  the  predominant 
medium,  followed  by  magazine/print  media.  Not 
surprisingly,  in  the  lowest  budget  cell  (Blue), 
considerably  more  reliance  is  placed  on  less 
expensive  media  (magazine/print  and  direct  mail). 
Furthermore,  the  Services'  decisions  to  eliminate 
electronic  media  in  Cells  Blue  and  Red  are 


consistent  with  advertising  theories  suggesting 
that  a  threshold  level  of  spending  (beyond  the 
limited  resources  available)  is  required.  In  Cell 
Red,  the  only  case  where  an  advertiser  (JRAP) 
has  a  higher  budget  than  in  the  control  cell,  a 
substantially  larger  percentage  of  the  national 
advertising  budget  is  allocated  to  television. 


Table  8 


Planned  Media  Expenditures  for  National  Advertising 
(Percent  of  Advertiser's  Cell  Budget) 


Cell 

Advertiser 

TV 

Radio 

Mag/ 

Print 

Outdoor 

JRAP 

78.6 

0.0 

20.0 

0.0 

White  A 

Services 

51.5 

15.4 

27.0 

0.5 

Total* 

56.9 

12.5 

25.6 

0.4 

JRAP 

74.4 

0.0 

24.6 

0.0 

White  A1 

Services 

51.5 

15.4 

27.0 

0.5 

Total* 

57.3 

11.2 

27.0 

0.4 

JRAP 

73.6 

0.0 

24.9 

0.0 

Blue 

Services 

0.0 

0.0 

82.0 

1.6 

Total* 

47.5 

0.0 

45.1 

0.5 

JRAP 

37.3 

0.0 

59.4 

0.0 

Green 

Services 

51.5 

15.9 

25.8 

1.2 

Total* 

50.7 

14.6 

28.0 

1.2 

JRAP 

88.0 

0.0 

11.2 

0.0 

Red 

Services 

0.0 

0.0 

81.1 

3.6 

Total* 

71.9 

0.0 

24.1 

0.6 

’Totals  may  not  add  due  to  rounding 

Timing  of  National  Advertising  Media 
Schedules 

Figure  2  depicts  planned  FY  1984  media 
schedules  for  JRAP  and  the  Services  by  month 


for  each  test  cell.  Essentially,  these  schedules 
demonstrate  that  when  use  of  a  medium  is 
planned,  there  is  little  difference  in  the  timing  of 
that  use  across  cells. 
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OFFICE  OF  THE  ASSISTANT  SECRETARY  OF  DEFENSE 

WASHINGTON  DC.  MMI 


MANSOWtft. 

scscsvt  srr  aim 

A  NO  LOGISTICS 

(Military  Porsoonol  4  Fores  Management) 


7  -  OCT  1983 


MMOIARDUM  FOE  ASSISTANT  SECKETAKT  OF  TBB  AMT  (M4IA) 

ASSISTANT  SECtETAKT  OF  TIE  RAFT  (NASA) 

ASSISTANT  SECKEIAH  OF  TIE  All  FOKCE  (MEA4I) 

SUBJECT:  Uses  Collection  for  ths  Departasat  of  Dsfooss  AdYsrtlslog  Mix  Tost 


This  aeaoraodua  forwards  ths  dots  eoilactlon  plan  for  ths  Dspsrtasot  of  Dsfsnss 
Advertising  Nix  Tost. 

Ths  plan  daflnss  ths  data  rsqulraasnts  and  raportlng  f ornate,  asdiua  and 
schsduls.  Ths  plan  la  divided  Into  two  parts  with  supporting  appsndlcss: 

Fart  I  -  Data  to  ba  provided  by  Advertisers  (Any,  Navy,  Air  Fores,  Marins  Corps 
and  Joint  Advertising  Frograa) 

Fart  II  -  Data  to  ba  furnlshsd  by  ths  Dsfsnss  Manpawur  Data  Csntsr  (DMDC) 

Appsndlx  A  -  Dstallsd  administrative  guldancs 

Appendix  B  -  Glossary  of  tarns  sad  dsflnltlons  of  data  olsasnts 

I  an  happy  to  acknowlsdgs  ths  eooparstlon  ws  have  received  froa  your  staff  In 
eoaplstlng  this  difficult  task.  Ws  will  eontlous  to  work  vary  closely  with 
your  staff  during  ths  tsst  psrlod.  My  point  of  eontaet  for  this  effort  Is 
Mr.  ton  Llverls  at  697*9267. 


E.  A. 


A.  Chavarria 
Llsutsnant  Gsnsral,  USAF 
Deputy  Assistant  Sserstary 
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PART  I  -  DATA  TO  BE  PROVIDED  BT  ADVERTISERS 


ITEM  1  -  RECRUITER  STAFF 


DATA  REQUIRED: 

Part  1.  Busbar  of  production  racrultara  by  aonth  by  Main  Station  for  FY 
1981.  FT  1982.  FY  1983. 

Part  2.  Nuabar  of  production  recruiters  by  aonth  by  ADI  for  Test  Cells 
Al.  B,  C,  and  D  (sea  Appendix  C  for  definition  of  test  cell 
ADIs)  for  period  of  test  starting  with  October  1983  data. 

Part  3.  Nuabar  of  new  recruiters  by  aonth  by  Main  Station  for  period  of 
test  but  starting  with  April  1983. 

REPORTING  FORMATS: 


1.  Handing:  Main  Station  Racruiter  Report  Historical  Data;  Service: 
Calendar  Year  and  Month 
Fields:  Main  Station  Naae  and  Nuabar 

Production  Recruiter  Count  (Enlisted,  NFS.  Active-Duty) 


2 .  Heading : 
Fields: 


ADI  Racruiter  Report;  Service;  Calendar  Year  and  Month 
ADI  Naae  and  Nuabar 

Production  Recruiter  Count  (Enlisted.  NPS.  Active-Duty) 


3.  Heading: 
Fields: 


New  Recruiter  Report;  Service;  Calendar  Year  and  Month 
Main  Station  Naae  and  Nuabar 

New  Recruiter  Count  (Enlisted.  NPS.  Active-Duty) 


REPORTING  MEDIUM: 


1.  Magnetic  Tape 

2.  Hard  Copy  Report 

3.  Hard  Copy  Report 

REPORTING  SCHEDULE: 

1.  Single  subalsslon  due  by  January  15.  1984. 

2.  Monthly  within  60  days  of  aonth' s  end  beginning  with  October  1983 
data. 

3.  April  1983  to  Septeaber  1983  data  due  by  Nov caber  15,  1983. 
Thereafter,  aonthly  within  60  days  of  aonth 's  end. 

RESPONSIBILITY: 

Each  Service  will  provide  the  data  directly  to  Wharton  Applied 
Research  Canter  (WARC). 
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ITEM  2  -  LOCAL  ADVERTISING 


DATA  REQUIRED: 

Part  1.  Summary  of  local  advertising  axpandituras  by  Main  Station  by 
month  for  7Y  1981.  1982  and  1983. 

Part  2.  Monthly  summary  of  local  advertising  expenditures  by  ADI  for 
test  cells  Al.  B.  C  and  D  for  period  of  test  starting  with 
October  1983  data. 


REPORTING  FORMATS: 


1.  Heading:  Main  Station  Local  Advertising  Report;  Service;  Calendar  Year 
and  Month 

Fields:  Main  Station  Name  and  Number 
Program  ID* 

Total  Net  Cost  of  Local  Advertising 

Net  Cost  of  Electronic  Advertising 

Net  Cost  of  All  Other  Forms  of  Local  Advertising 


2.  Heading:  ADI  Local  Advertising  Report;  Service;  Calendar  Year  and  Month 
Field:  ADI  Name  and  Number 

Program  ID* 

Total  Net  Cost  of  Local  Advertising 

Net  Cost  of  Local  Electronic  Advertising 

Net  Cost  of  All  Other  Forms  of  Local  Advertising 


REPORTING  MEDIUM:  Hard  Copy  or  Magnetic  Tape 


REPORTING  SCHEDULE: 


1.  Single  submission  due  January  15.  1984. 

2.  Enlisted.  NPS.  active-duty  program  data  to  be  reported  monthly  within 
60  days  of  month's  end  starting  with  October  1983  data.  Each 
month's  report  should  contain  updates  or  corrections  as  necessary  to 
previous  month's  data.  All  other  program  data  may  be  reported 
quarterly  within  60  days  of  quarter's  end  beginning  with  first 
quarter  FY  1984  data. 

RESPONSIBILITY: 

Each  Service  will  provide  data  directly  to  Wharton  Applied  Research  Center. 


*Programs  Include:  (01)  Enlisted.  NPS,  Active-Duty:  (02)  Active  Duty 
Officer;  (03)  Reserve-Duty  Officer;  (04)  Reserve-Duty  Enlisted;  (05)  Prior 
Service;  (06)  Special  Programs 


ITEM  3  -  RATIONAL  LEAD  SUMMAKT 


DATA  REQUIRED: 

Parc  1.  Nuaber  of  grots  tod  qualified  anil scad ,  NFS,  active-duty  na clonal 
advarclalog  laada  ganaracad  aaeh  aonch  by  ADI  for  cht  test 
parlod  and  ona  qua rear  btyood. 

Parc  2*  Crlcarla  uaad  by  Services  for  qualifying  loads  (a.g.  ago,  aducaclon 
ate). 


REPORTING  FORMATS; 


1.  Haadlng:  National  Advarclalog  La ad  Suaaary  Raporc;  Calendar  Yaar 
and  Month. 

Flalda:  Service/ Joint 

ADI  Naaa  and  Nuaber 
pros a  Load  CounC  . 

Quail f lad  Load  CounC 


2.  Haadlng:  Load  Quallf lcadon  Crlcarla;  Service. 
Plaids:  Ago 

Educaclon 

Ochar  (a.g.  dupllcaea  Inquiries) 


REPORTING  MEDIUM: 


1.  Magnetic  Tape 

2.  Hard  copy 


REPORTING  SCHEDULE: 

1.  Monthly  vlchln  60  days  of  aonth'e  and  starting  vlch  October  1983  data 
and  ending  with  the  third  aonch  after  coaplecion  of  che  test. 

2.  Criteria  io  use  on  1  October  1983  and  any  changes  since  chat  date 
are  due  30  Noveaber  1983.  Changes  in  criteria  afetr  30  Noveaber 
should  be  reported  as  changes  occur. 

RESPONSIBILITY: 

1.  Each  Service  and  the  Joint  Recruiting  Advertising  Prograa  Director 
will  provide  data  dlrecdy  co  Wharton  Applied  Research  Center. 

2.  Each  Service  will  provide  daea  dlrecdy  to  Wharton  Applied  Research 
Center  wich  a  copy  furnished  0ASD(MRA&L)(MP6FM)(AP). 
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REM  4  -  NATIONAL  ADVERTISING  EXPENDITURES 
DATA  REQUIRED: 


Listing  of  national  advertising  expenditures  by  appearance  or  Insertion 
by  aonth  for  the  period  FT  1981  through  the  period  of  the  test. 

REPORTING  FORMATS: 

Heading:  .  National  Advertising;  Service/ Joint ;  Calendar  Month;  Calendar 
Tear;  Program  ID* 

Fields:  Lead  Generation  Code  (Does  sd  contain  a  lead  generation  aechanlsa) 
Magazine  or  Newspaper  Name  or  Code 
Magazine  or  Newspaper  Edition  and  Insertion  Size 
Magszlne  Issue  Date 
Magazine  or  Newspaper  Net  Cost 
Television  Network  or  Spot  Code 
Television  Program  and  Statlon(s)  Exhibiting  Ad 
Television  Ad  Length;  Date;  Exact  Time  of  Airing; 

Daypart  or  Daypart  Code 
Television  Ad  Nee  Cost 
Radio  Network  or  Spot  Code 
Radio  Program  and  Station(s)  Airing  Ad 
Radio  Ad  Length;  Date;  Exact  Time  of  Airing; 

Daypart  or  Daypart  Code 
Radio  Ad  Net  Cost 

Billboard  ADI  Number;  Date  Visible;  Ad  Length  of  Run 
Billboard  Net  Coat 

Direct  Mall  Audience  Identifier  Code 

Direct  Mall  Size  of  Mailing  (Gross  Pieces  Mailed)  and  Mailing  Date 
Direct  Mall  Net  Cost 

ADI  Name;  ADI  Number;  Direct  Mall  Pieces  Sent  to  ADI 
REPORTING  MEDIUM:  Magnetic  Tape 
REPORTING  SCHEDULE: 

Each  advertiser  will  provide  a  file  layout  diagram  and  coding  sheets  to 
VARC  for  review  by  November  IS,  1983.  Enlisted,  NPS,  active-duty  program 
data  for  the  period  of  the  test  beginning  with  October  1983  data  will  be 
reported  monthly  within  60  days  of  month's  end.  Data  for  programs  other 
than  the  enlisted,  NPS,  active-duty  program  for  the  period  of  the  test 
may  be  reported  quarterly  within  60  days  of  quarter's  end  beginning  with 
first  quarter  FT  1984  data.  FT  1981  and  FT  1982  data  for  all  programs  is 
due  in  a  single  submission  on  January  IS,  1984.  FT  1983  data  for  all 
programs  is  due  in  s  single  submission  on  February  28,  1984. 

RESPONSIBILITT: 

Each  Service  and  the  Joint  Recruiting  Advertising  Program  Director  will 
provide  data  directly  to  Wharton  Applied  Research  Center. 


*  Programs  include:  (01)  Enlisted.  NPS,  Active-Duty.  (02)  Active-Duty  Officer; 
(03)  Reserve-Duty  Officer;  (04)  Reserve-Duty  Enlisted;  (OS)  Prior  Service; 
(06)  Special  Programs. 


ITEM  S  -  MISSION  OK  GOALS  AND  RECRUITING  POLICIES 


DATA  REQUIRED: 

Part  1.  Enlisted,  NPS,  active-duty  contract /accession  mission 

for  each  aain  station  by  aonth  for  the  period  of  FT  1981 
through  the  and  of  the  test. 

Part  2.  A  narrative  description  of  major  recruiting  policies  In  effect 
during  the  test. 

REPORTING  FORMATS: 


s' 

< 

X 

s 

a. 


t 

r, 

P 

► 

, 

* 

? 


1.  Reading:  Mission  Report;  Service;  Calendar  Tear  and  Month 
Fields:  Main  Station  Naae  and  Nun bar 

Enlisted,  NPS,  Active-Duty  Contract  Mission  (sex;  educatlon-HSDG . 

HS  senior.  NHSDG;  AFQT  1-3 A.  3B,  4) 

Enlisted,  NPS,  Active-Duty  Accession  Mission  (sex;  educatlon-HSDG. 
HS  senior.  NHSDG;  AFQT  1-3A.  3B.  4) 


2. 


Heading:  Recruiting  Policies;  Service;  Calendar  Tear  and  Quarter 
Topics:  The  following  topics  should  be  addressed  in  this  report: 


-  Enlistment  Standards  (education  and  AFQT  eligibility 

standards  and  national  goals) 

-  DEP  Policies  and  Conatralnts  (DEP  slse  and  nix  goals;  DEP 

eligibility  restrictions,  naxlaua  tine  restrictions) 

-  Fiscal  Policies  and  Constraints  (e.g.  vehicle  alleage 

and  applicant  travel  restrictions) 

-  Marketing  Strategies  and  Priorities  (e.g.  trade-offs 

between  PS  and  NPS  and  male  and  feaale  accessions) 

-  Enlistment  Options  (e.g.  Enlistment  Bonus.  Educational 

Benefits,  terms  of  enlistment 

-  Enlistment  Processing  Policies  (e.g.  MEPS  operating  hours) 


REPORTING  MEDIUM: 


1.  Magnetic  Tape  or  Hard  Copy 

2.  Hard  Copy 

REPORTING  SCHEUDLE: 

1.  Data  for  FT  1981  through  FT  1983  due  December  13,  1983.  Data  for  first 
quarter  FT  1984  due  December  IS,  1983.  Data  for  each  subsequent  quarter 
will  be  reported  within  the  first  week  of  the  start  of  the  quarter  except 
for  the  Navy  which  will  report  monthly  by  the  middle  of  the  month 
starting  with  January  1984. 

2.  Quarterly  starting  with  the  first  quarter  of  FT  1984  and  ending  with 
last  quarter  of  test.  Data  for  each  quarter  will  be  reported  within 
13  days  of  quarter's  end. 

RESPONSIBILITY 

1.  Each  Service  will  provide  data  directly  to  Wharton  Applied  Research 
Center. 

2.  Each  Service  will  provide  the  data  to  Wharton  Applied  Research  Center 
with  a  copy  to  OASD  (MRA4L)  (MPAFM)  (AP). 
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XTZM  6  -  TEST  COSTS 


DATA  KEQOIIEP: 

Parc  1.  Updates  of  actual  and  projaetad  coses  for  advartlslnf  ralacad  to  the 
DoD  Advertising  Mis  Test.* 

Part  2.  Updates  and  explanations  of  costs  for  advertising  aedla  purchasing 

loaf fl cieneies  ( cut-out ,  block-out,  buy-up  and  non-relabursable  costs) 
and  advertising  data  collection  resulting  f roa  the  test. 


REPORTING  FORMAT  (see  Appendix  D): 


1.  Heading:  DoD  Advertising  Mix  Test  Costs;  Service/ Joint ;  Reporting 
Data  (Calendar  Tear,  Month  and  Day) 

Fields:  Advertising  Period  Covered** 

FT  Appropriations  (e.g.  FT  1983  funding.  FT  1984  funding) 

National  Media  Placeaent  Costs  (TV,  radio,  nagaslne,  other  aedla) 

National  Media  Production  Costs 

Direct  Mail  Costs 

Local/Regional  Advertising  Costs 

Other  Costs  (Specify) 


2 .  Heading : 


Fields : 


Topics : 


Advertising  Media  Purchaalng  Inefficiencies  end  Data 
Collection  Costs;  Service/ Joint;  Reporting  Date  (Calendar  Tear, 
Month  and  Day) 

Advertising  Period  Covered** 

FT  Tear  Appropriations 
Advertising  Data  Collection  Costs 
Test  Cell  (B-Blue.  C-Green,  D-Red) 

Special  Costs  (cut-out.  black-out,  buy-up  snd  non- 
relabursable  costs) 

Explain  Special  Costs  snd  Advertising  Data  Collection  Costs. 


REPORTING  MEDIUM: 


Hard  Copy 


REPORTING  SCHEDULE: 

142:  Quarterly  starting  vlth  advertising  for  first  quarter  of  FY  1983  and 

for  the  first  9  aonths  of  the  test.  Data  will  be  reported 
within  15  days  of  quarter's  end.  One  special  report  estlaatlng  test 
costs  for  FT  1984  advertising  will  be  subaltted  by  Noveaber  30,  1983. 

BESPOHSIBILITT: 

Each  Service  and  the  Joint  Recruiting  Advertising  Prograa  Director  will 
provide  data  directly  to  OASD  (MRA4L)(MP4FM)(AF). 


*Includes  costs  for  all  categories  Identified  In  "Foraat  V"  for  enlisted  NFS. 
active-duty  advertising  for  test  purposes.  It  Includes  costs  for  black-outs, 
cut-outs,  buy-outs  and  non-relabursable  costs. 

**Data  will  be  reported  for  advertising  covering  two  periods:  Advertising  froa 
the  start  of  the  cost  to  the  start  of  the  currant  aonth;  advertising  froa  the 
start  of  the  test  to  the  start  of  sixth  aonth  after  the  current  aonth. 
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RIM  7  -  MAIM  STATION  -  COUNTY  CROSSWALK 


3UIRED: 


A  list  of  eouotioo  included  la  oaeh  Mala  Statloa  for  the  period  FT  1981 
through  eha  aad  of  tha  east.  (Kota  change*  and  else  of  changes  since 
October  1,  1980). 

REPORTING  FORMAT: 

Heading :  Main  Station-County  Crosswalk;  Service.  Effective  Date. 
Fields:  FIPS  State  Code 
FIPS  County  Code 
Main  Station  Naae  and  Nuaber 


REPORTING  MEDIUM: 


Magnetic  Tape 
REPORTING  SCHEDULE: 

Currently  available  crosswalk  aad  any  changes  dating  froe  FT  1981  forward 
should  be  reported  in  a  one  tlae  submission  due  Koveaber  15.  1983.  Any 
changes  which  occur  froa  the  currently  available  status  should  be 
reported  with  this  data  or  as  changes  occur.  These  changes  aay  be 
submitted  in  hard  copy  fora  if  substantial  tlae  will  be  required  for  data 
processing. 

RESPONSIBILITY: 

Each  Service  will  provide  data  directly  to  Wharton  Applied  Research 
Center. 
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PART  IZ  -  DEFENSE  MANPOWER  DATA  CENTER 

The  data  lteaa  ata  r  aquas  tad  froa  DMDC.  Services  will  work  with  DMDC  to 
as aura  that  data  eleaents  dascrlbad  balow  ara  avallabla  at  DMDC  and  ara 
adaquata  for  evaluation  purposes. 


ITEM  1  -  ENLISTED  FIRST  APPLICANTS;  ENLISTED  CONTRACTS;  INDIVIDUALS  IN  DEP 
DATA  REQUIRED: 

1.  Nuaber  of  anllstad  first  applicants  by  aonth,  Sarvlca/DoD  and  ADI  for 
FT  1981  through  FT  1984. 

2.  Nuabar  of  anllstad  contracts  by  aonth,  Sarvlca/DoD  and  ADZ  for  FY 
1981  through  FY  1984. 

3.  Nuabar  of  Individuals  in  tha  DEP  froa  aach  ADI  for  aaeh  Service 
starting  with  Septeaber  1980  and  ending  with  Septeaber  1984. 


REPORTING  FORMAT: 

1.  Reading:  Nuabar  of  Enlisted  First  Applicants;  Sarvlca/DoD;  Calendar 

Year  and  Month. 

Fields:  ADI  Dim  and  Nuabar 

Total  First  Applicants  by  Sax  and  Educatlon/AFQT  quality  indices 

2.  Heading:  Nuabar  of  Enlisted  Contracts;  Servlee/DoD;  Calendar  Year 

and  Month 


Fields:  ADI  Name  and  Nuabar 

Total  (NFS  plus  PS)  Contracts 

NPS  Contracts  by  Sex  and  Educatlon/AFQT  quality  Indices 
PS  (aale  plus  feaale)  contracts 

3.  Heading:  Nuaber  of  Individuals  in  DEP;  Servlce/DoD;  Calendar  Year 
and  Month 


Fields:  ADI  Naas  and  Nuaber 

Total  Nuaber  In  DEP  by  Sex  and  Educatlon/AFQT  quality  indices 
Total  Nuaber  Scheduled  for  Active-Duty  Next  Month  by  Sex  and 
Educatlon/AFQT  quality  indices 
DEP  attrition  by  Sex,  Educatlon/AFQT  quality  indices, 
contract  data  and  ship  date. 


REPORTING  MEDIUM: 


Magnetic  Tape 
REPORTING  SCHEDULE: 


FY  1981  and  FY  1982  Data  Due  November  30,  1983;  FY  1983  Data  Due 
January  31,  1984;  Data  for  period  of  test  due  monthly  within  120  days 
of  aonth' s  end  beginning  with  October  1983  data 

RESPONSIBILITY: 


DMDC  (via  West  Coast  POC)  using  HEP COM  source  data  will  provide  data 
directly  to  Wharton  Applied  Research  Center. 


ITEM  2  -  TOOTH  ATITUDE  TRACKING  STUDY  DATA  (TATS) 


DATA  REQUIRED: 

Mala  raaponaaa  in  Fall  1981,  Fall  1982.  Fall  1983  and  Fall  1984  TATS. 
REPORTING  FORMAT 

Aa  formatted  on  existing  magnetic  tape. 

REPORTING  MEDIUM: 

Magnatlc  Tape 
REPORTING  SCHEDULE: 

Fall  1981  and  Fall  1982  TATS  responses  due  November  30.  1983 
Fall  1983  TATS  responses  due  February  28,  1984 
Fall  1984  TATS  responses  due  February  28.  1985 

RESPONSIBILITY: 

DMDC  (via  East  Coast  POC)  will  provide  data  directly  to  Wharton  Applied 
Research  Center. 


ITEM  3  “  RECRUITING  ENVIRONMENT 
DATA  REQUIRED: 

Selected  economic  and  demographic  data  available  from  DMDC. 
requirements  are  to  ba  defined  by  October  31,  1983. 

REPORTING  FORMAT: 

To  ba  determined 

REPORTING  MEDIUM: 

To  be  determined 

REPORTING  SCHEDULE: 

To  be  deteralned 

RESPONSIBILITY; 


Specific 


DMDC  (via  Eaat  Coast  POC)  will  provide  the  data  directly  to  Uharton 
Applied  Research  Center. 


APPEXDIX  A  -  ADKXIXSTIATXFB  6VZ0AKI 


1*  OASD  (MkAAL)  (A?)  point:  of  contact  -  LTC.  John  Ford  A  227*9267. 

Address:  OASD  (MKAAL)  (MFAFM)  (A?) 

CAtta:  LTC.  Joba  Ford),  la.  2B271 
Tho  Pentagon 
Washington.  D.C.  20301 

2.  Defense  Manpower  Data  Center  points  of  contact:  East  Coast  ~  Mr.  Paul 
lichens  (202)  696*5837;  West  Coast  *  Ms.  Helen  Hagen  (608)  375*2111. 


BAST  COAST 

Address:  Defense  Manpower  Data  Center 
(Attn:  Mr.  Paul  lichens) 

4th  Floor 

1600  Wilson  Boulevard 
Arlington,  FA  22209 


WMT  COAST 

Defense  Manpower  Data  Center 
(Attn:  Ms.  Helen  Hagen) 

550  El  Ceaino  Estero 
Monterrey,  CA  93940 


3.  Wharton  Applied  Research  Center  point  of  contact  *  Ms.  Judith  Mauer 
(215)  898-4768. 


Please  send  all  data  elements  which  indicate  direct  provision  to  the 
Wharton  Applied  leseareh  Center  to: 

Wharton  Applied  leseareh  Center 
Joint  Services  Advertising  Project 
Suite  100 
3508  Marhet  Street 
Philadelphia,  PA  19104 

4.  Clearly  label  (externally)  each  data  su (mission,  file  layout  diagram  or 
coding  sheet  submitted  to  include: 

e  Service  or  source 

e  Data  item(s)  covered  (e.g.  national  Advertising-enlisted,  non-prior 
service,  active  duty  advertising  progrsa). 

e  Period  covered  (e.g.  *  Dec.  1983) 

e  Point  of  contact  of  service  or  souree  organisation. 

5.  Wharton  Applied  leseareh  Center  will  achnowledge  in  writing  the  receipt  of 
each  data  submission  to  the  source  organisation  point  of  contact  within  48 
hours  of  receipt. 

6.  Because  of  the  large  number  of  megnetic  tapes  to  be  processed  the 
following  standard  tape  specifications  are  required: 

e  EBCXDIC  (Standard  IBM) 

e  1600  hits  per  inch  (no  compressed  data  please) 
e  9  trach 

e  80  characters  per  record 
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APPENDIX  B  -  GLOSSARY  OF  TEIMS  AMD  DEFINITIONS  OF  DATA  ELEMENTS 

API:  Arts  of  dominant  influonct  u  dofinod  by  Arbitron  Corporation.  An  API 
la  coapoatd  of  eountioa  a  plurality  of  whose  raaidonta  watch  television 
eaanatint  f row  the  aaae  location.  For  the  purposes  of  the  Joint 
Services  'Advertising  Experiment  APIs  are  strictly  defined  by  the  1981-82 
Arbitron  boundaries.  API  naaes  and  nuabers  will  correspond  to  the 
Arbitron  1981-82  naae  and  nuaber  convention.  All  Pate  collected  on  an 
API  basis  will  include  data  only  froa  those  counties  which  are  defined 
by  the  1981-82  Arbitron  boundaries. 

apt?  The  API  on  which  the  billboard  ad  displayed  is  located. 

^rymnABO  date  VISIBLE:  Defined  as  the  date  that  a  billboard  ad  is  actually 
affixed  to  the  billboard. 

datpabt ;  Defined  as  the  section  of  the  day  in  which  an  ad  appeared.  This 
itea  can  be  aeasured  and  reported  on  whatever  basis  the  agency  or 
Service  generally  use.  The  measure  for  each  aediua  should  be  kept 
consistent  throughout  the  test  period  and  should  be  clearly  indicated  on 
coding  sheets  provided  with  the  data  subaission. 

PEP:  Delayed  entry  program  is  composed  of  individuals  who  have  signed 

enlistment  contracts  but  have  not  yet  reported  for  active-duty  training 
or  aasignaent.  Counts  of  PEP  personnel  will  account  for  all  individuals 
in  PEP  as  of  the  end  of  a  given  calendar  aonth. 

DIRECT  MAIL  AUDIENCE  IDENTIFIER  CODE:  Defined  as  a  code  to  indicate  the 
audience  to  which  direct  aail  is  directed.  Indicate  target  market, 
influencer  and  any  other  specific  audiences  which  aay  be  appropriate. 
Please  clearly  identify  this  coding  scheae  on  coding  sheets  submitted 
and  aaintain  code  consistency  throughout  the  test. 

DIRECT  MAIL  MAILING  DATE:  Defined  as  the  date  that  direct  mail  advertising  is 
first  provided  to  the  Post  Office  for  handling  and  delivery. 

ELECTRONIC  ADVERTISING:  Defined  in  this  document  as  paid  advertising 
appearing  on  either  radio  or  television. 

mffi  OP  AIRING :  Defined  as  the  hour,  a.m.  or  p.m.,  minute,  and  second 
when  an  ad  began  its  appearance.  For  ads  appearing  in  multiple  time 
sones  use  Eastern  Time. 

FIRST  APPLICANT:  Defined  as  an  individual  taking  the  non-institutionel  Armed 
Services  Vocational  Aptitude  Battery  test  sequence  for  the  first  time. 
Counts  of  first  applicant  personnel  will  account  for  all  individuals  who 
have  taken  the  ASVAB  during  a  given  calendar  aonth. 

FULFILLED  LEAD:  Defined  as  a  national  advertising  lead  to  whoa  material  or 
information  has  been  provided  and  whose  naae  has  been  or  could  be 
forwarded  to  a  Service  recruiting  organisation  for  follow-up. 
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nmATiai  cont;  a  coda  to  iadicoto  whether  a  particular  ad  eoataiaa 
aay  of  tha  following: 

-  Toll  froo  tolophoao  auaber 

-  Business  reply  card 

-  On -page  coupon 

-  Poat  card 

-  Self-addressed  envelope 

maniut:  Local  advert ia lag  la  that  advert iaiag  which  ia  purchased 
or  authorisod  at  a  local  level.  It  ia  goaorally  adaiaiatered  through 
the  recruit iag  Maia  Statioa  cowaad  atructuro. 


1  OK  «W»Am  IMSEETIOM  SIZE:  Defiaed  aa  the  aiae  of  a  aagatiae  or 
newspaper  iaaertioa  aeaaurod  ia  whatever  ataadard  foraat  the  reporting 
Service  gcaerally  uaea.  All  newspaper  iaaertioaa  ahould  be  aeaaured  on 
the  saae  baaia.  All  aagaaiae  iaaertioaa  ahould  be  aeaaured  on  the  eaae 
baeie. 


BE  Ot  MEWBPAPEE  I8SPE  BATS:  Defiaed  aa  the  date  the  publication  ia 
firat  available  for  sale  to  the  general  public.  (If  there  ia  ao 
general  public  aale  then  the  nailing  date  of  the  firat  aubacriptiou 
piece  of  the  publication  iaaue  will  be  need. 


maim  mnaii  Any  Dietrict  Bacruitiag  Coaaaad,  levy  Eecruitiag  Diatrict. 
Marine  Corps  lecruitiag  Station,  or  Air  Force  Squadron. 

mattobai.  ADVEETISTwe  i.kai>«  Defiaed  here  aa  the  provision,  on  request  of  an 

individual  respondeat,  of  aaterial  about  enlisted  prograa  opportunities 
or  featuree.  The  request  for  additional  iaforaatioa  ia  generated  by: 

-  Calling  a  toll-free  telephone  nuaber 

~  Sending  in  a  reply  card  or  on -page  coupon 

-  Mailing  back  a  poat  card  or  envelope 

which  appear  in  or  are  contained  in  national  advertising  presentations. 

MEW  lECEOITEE :  Service  recruiting  personnel  who  are  beginning  the  first  field 
recruiting  assigaaent  of  their  first  tour  of  recruiting  duty  and  who 
have  or  are  expected  to  have  an  enlisted,  EPS,  active-duty  aission  (or 
goal)  or  aiss ion-part. 

MET  COST:  Defined  ia  this  docuaent  aa  the  coat  expended  to  all  vendors  for 

paid  advertising  appearances  net  of  all  discounts,  rebates,  allowances, 
and  advertising  agency  fees,  coaaiss ions,  or  allowances. 

T 1P1 *  Where  ealled  for  ia  this  docuaent  please  use  the  following 
prograa  codes: 

(01)  Enlisted,  MPS,  Active-Duty  Prograa 
(02)  Active-Duty  Officer  Prograa 
(03)  Keserve-Duty  Officer  Prograa 
(04)  leaerve-Duty  Enlisted  Prograa 


*  s1  V  V  O  SW  v*  ■»"  ».*  V  •  AM*  •*,  'V'  *  e  *. 


(05)  Prior  Sortie*  Progr 
(06)  Special  Programs 


PBODOCTIOB  1BC1PITBE:  Sortie**  recruiting  personnel  who  hare  an  enlisted , 

KPS,  ac tire-duty  mission  (or  goal)  or  mission-part.  A  recruiter  will 
be  defined  ae  being  in  a  market,  ADZ,  or  Main  Station  as  of  the  first 
calendar  day  of  a  given  month. 

canto  An  LEMGTH:  Defined  as  the  duration  of  a  radio  ad  in  seconds. 


1ADI0  METWOBK  Qg  SPOT CODE:  A  code  to  identify  the  radio  network  on  which  a 
radio  ad  was  purchased  and  appeared.  Alphabetic  codes  (e.g.  MIC,  CBS, 
ABC)  are  acceptable.  If  the  ad  was  not  purchased  and  distributed  by  a 
network  then  use  a  spot  code  to  indicate  that  the  ad  was  purchased  on  a 
spot  basis.  Alpha  (e.g.  sss)  or  numeric  (e.g.  999)  codes  are 
acceptable. 


IADI0  mum  aauag  AM:  Defined  as  a  listing  of  all  stations  which 

exhibited  a  given  radio  ad.  An  alternative  form  of  data  submission  is 
acceptable  for  this  item  as  follows:  A  service  can  elect  to  provide  a 
comprehensive  and  complete  list  of  all  statioma  affiliated  with  each 
network  used.  In  this  case  the  Service  will  list  as  stations  only 
those  stations  where  the  ad  did  not  air.  Stations  where  the  ad  did  not 
air  will  be  listed  whether  the  ad  did  not  clear  on  a  specific -station 
ordered  or  because  a  specific  station  was  cut-out  or  blacked-out  for 
purposes  of  this  or  say  other  test.  If  this  form  of  data  submission  is 
elected  it  is  the  responsibility  of  each  service  to  update  the  list  of 
affiliated  stations  for  each  network  as  necessary. 


,8IW  AP  LBKrIB- 


Defined  as  the  duration  of  a  television  ad  in  seconds. 


TBLEVISIOM  METWOIK  OB  SPOT  CODB:  A  cod*  to  identify  the  television  network  on 
which  a  television  ad  was  purchased  sad  appeared.  Alphabetic  codes 
(e.g.  MBC,  CBS,  ABC)  are  acceptable.  If  the  ad  was  not  purchased  end 
distributed  by  a  network  then  use  a  spot  code  to  indicate  that  the  ad 
was  purchased  on  s  spot  basis.  Alpha  (e.g.  sax)  or  numeric  (e.g.  999) 
codes  are  acceptable. 


TELEVISIOH  STATIOHS  EXHIBITIHG  ADS:  Defined  as  a  listing  of  all  stations 
which  exhibited  a  given  television  ad.  An  alternative  form  of  data 
submission  is  acceptable  for  this  item  ss  follows:  A  Service  can  elect 
to  provide  a  comprehensive  and  complete  list  of  all  stations  affiliated 
with  each  network  used.  In  this  case  the  Service  will  list  as  stations 
only  those  stations  where  the  ad  did  not  air.  Stations  where  the  sd 
did  not  air  will  be  listed  whether  the  sd  did  clear  on  a  specific 
station  ordered  or  because  a  specific  station  was  cut-out  or  blacked- 
out  for  purposes  of  this  or  any  other  test.  If  this  form  of  data 
submission  is  elected  it  is  the  responsibility  of  esch  service  to 
update  the  list  of  affiliated  stations  for  esch  network  ss  necessary. 


APPENDIX  C  -  ADXs  BT  TEST  CELLS 


CELL  A  (White) 

Albany ,  GA 

Albany- Schenectady,  MY 

Albuquerque 

Alexandria i  LA 

Alpena 

Aaarillo 

Annie con 

Ardaore,  ASA 

Atlanta 

Augusta 

Austin,  TX 

Bakersfield 

Baltiaore 

Bangor 

Baton  Rouge 

Beauaont-Port  Arthur 

Billings-Hard in 

Binghsaton 

Birainghaa 

Bluefie ld-Beck ley 

Boise 

Boston 

Bowling-Green 

Bristol-Kingport 

Buffalo 

Burlington-Plattaburgh 

Cedar  Rapida-Vaterloo 

Charleston,  SC 

Charleston-Runt ington 

Charlotte 

Chattanooga 

Chicago 

Chico-Redding 

Cincinnati 

Clarksburg-Weston 

Colorado  Springs 

Coluabia,  SC 

Co luab is- Jeff  er son  City 

Coluabus,  CA 

Co luabus-Tupc lo 

Corpus  Christi 

Davenport -Rock  Island 

Dayton 

Deo  Koines 

Dothan 

Duluth-Superior 
El  Centro-Yuaa 
Elaira 
Erie 


Eureka 

Fargo 

Faraington 
Flint-Saginaw 
Florence,  SC 
Fort  Myers-Naplee 
Fort  Wayne 
Fresno 

Grand  Rapids-Kalaaazoo 
Great  Falls 
Green  Bay 

Greensboro-Wineton 

Greenville-Spartanburg 

Greenvood-Greenville 

Harrisonburg 

Hartford-New  Haven 

Helena 

Houston 

Huntsville-Decatur 
Idaho  Falls-Pocatello 
Jackson,  Ml 
Johns ton-Altoona 
Joplin-Pittsburg 
Kansas  City 
Knoxville 

La  Crosoe-Eau  Claire 
Lafayette,  LA 
Lake  Charles 
Lansing 
Las  Vegas 

Laurel-Hat  t ie  sbur g 
Lexington 

Liaa 

Lincoln-Has tings 

Little  Rock 

Los  Angeles 

Louisville 

Lubbock 

Macon 

Madison 

Marquette 

MeAllen-Brownsville 

Medford 

Meridian 

Miles  City-Glendsle 

Milwaukee 

Minot-Bisaark 

Missoula-Butte 

Mobile-Pensacola 


Monro e-Il  Dorado 
Montgomery 
Jteshville 
■m  York 

■orfolk-Port  smooth 
lortb  Platt* 
Odessa-Midland 
Oklahoma  City 
Omaha 

Orlaado-Daytona 
Ottumwa-Kirksville 
Paducah-Cap*  Girardea 
Palm  Spring* 

Panama  City 

Parkarabnrg 

Paoria 

Phoenix 

Pittsburgh 

Port land-Po land 

Presque  Zsla 

Providence-Mew  Bedford 

Quincy-Hannibal 

■aleigh-Durbaa 

tap id  City 

Baao 

riehmond 

Koanoka-Lynchburg 
Rochester,  R 
Boebester-Mason 
Boekford 

lo swell 

Sacremento-Stock 
St.  Joseph 
St.  Louis 
Salinas-Monterey 
Salisbury 
Salt  Lake  City 
San  Diego 
San  Franc iaeo 
Santa  Barbara-SA 
Sarasota 
Savannah 
Selma 

Shreveport-Tsxarkana 
Sioux  City 
Sioux  Palls-Mitehell 
Soputb  Bend-Elkhart 
Syracuse 
Tallahassee 
Tampa-St  Peters 
Terre  laute 
Toledo 


Topeka 

Traverse  City-Cadillsc 

Tucson 

Thscaloosa 

TWin  Palis 

Utica 

Victoria 

Washington,  DC 

Watertewn-Carthage 

Wausau-Bhinelander 

Whelliag-Steubcnville 

Wichita-Butchins 

Wichita'  Palls-Lavton 

Wilkes  Barre-Scranton 

Wilmington 

Takima 


CELL  A-l 

Atlanta 

Buffalo 

Charlotte 

Chico-kedding 

Cincinnati 

Corpus  Christ i 

Davenport-Bock  Island 

Eureka 

Houston 

Hunt  sv il le-Dec atur 

Idaho  Falls-Pocatello 

Jopl in-Pitt  sburgh 

Kansas  City 

La  Crosse-Eau  Claire 

Laure 1-Hat t iesburg 

Lexington 

Macon 

Milwaukee 

Paducah-Cape  Cirardeal 

Baleigh-Durham 

Sacramento-Stockton 

St.  Joseph 

Sa 1 inas-Monterey 

San  Francisco 

Sioux  Falls-Mitchell 

Topeka 

Tucson 

Victoria 

Wausau-Rhineland 

Wilkes  Barre-Scranton 

Takima 


D-18 


CELL  1  (Blue) 


CELL  D  (lad) 


Alexandria,  MM 

lead 

Biloxi^Sulfport 

Dallas-Ft.  North 

Casper-Eiverton 

Detroit 

Cheyenne 

11  Paso 

Cleveland 

Butene 

Denver 

Gainesville 

Evansville 

Barr iaburg-Tork 

Grand  Junction 

Jacksonville 

Greenville- Nev  Bern 

Portland,  OK 

Jackson,  IN 

San  Angelo 

Jonesboro 

Spring fie Id,  MO 

Laredo 

Tyler 

Mankato 

Vaco-Tenple 

Menphis 

CELL  C  (Green) 

Minneapolis-St.  Paul 
Nev  Orleans 

San  Antonio 

Columbus ,  OH 

Ft.  Snitb 

Indianapolis 

Lafayette,  IN 

Miami 

Philadelphia 

Scattle-Tacoma 

Tulsa 

Vest  Pain  Beach 

Zanesville 

Youngstown 

APPENDIX  D  -  FORMAT  FOR  SUBMITTING  TEST  COSTS 


I.  Advertiser  (e.g.  Army) 

II.  Reporting  Dete  (Calender  Teer.  Month.  Day) 

III.  Actual  end  Projected  Advertising  Coots*  (by  FT  Approprletion) 


Cost  Catego 


Period  1  (Completed) 
(e.g.  First  quarter  FT  1984) 
FT  1983  FT  1984 
Funds  Fuads  Total 


Period  2  (Projected)  ^ 

(e.g.  First  3  quarters  FY  19"4  J 
FT  1983  FT  1984  V 

Funds  Funds  Total  > 


s 


National  Media  Placeaent 


(Television) 

(Radio) 

(Magaslne) 

(Other  Media) 

National  Media  Production 
Direct  Mall 

Local/Reglonal  Advertising 
Other  (Specify) 

Total 

IV.  Special  Costs  -  Provide  end  explain  coots  for  cut-outs,  bleck-outs  buy-ups. 
and  aon-relabursable  costs  by  treatment  cell  (Blue,  Green,  Red  Cells), 
and  FT  appropriation  for  both  periods.  Also  Identify  and  explain  advertising 
data  collection  costs  uniquely  resulting  fron  the  test. 

*  All  enlisted,  NPS.  active-duty  advertising  costs  Identified  by  category  In 

"Fornat  V*  for  test  purpoaes  Including  costs  for  cut-outs,  blsck-outs  and  buy-ups 
and  non-relnbursable  costs. 
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of  Naval  Research  Contract  N00014-83-C-0663. 
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EXECUTIVE  SUMMARY 


BACKGROUND 

The  primary  purpose  of  the  FY  1984  Advertising 
Mix  Test  is  to  help  the  Department  of  Defense 
determine  the  optimum  budget  level  for  DoD 
recruiting  advertising  and  the  best  mix  of  Joint 
and  Service-specific  programs.  Directed  by  the 
Secretary  of  Defense,  the  test  varies  the  level  and 
mix  of  advertising  expenditures  in  accordance 
with  a  controlled  experimental  design.  The 
design  has  been  implemented  in  four  matched 
groups  of  markets  and  a  detailed  market  level  data 
base  is  being  assembled  for  subsequent  analysis. 
As  the  prime  contractor,  Wharton  Applied 
Research  Center  (WARC)  is  responsible  for 
providing  a  series  of  reports  documenting  the 
planning,  implementation,  data  collection,  and 
analysis  phases  of  the  year-long  test  This  report 
discusses  the  development  of  a  criterion  for 
evaluating  the  advertising  test.  Although  pre¬ 
viewed  in  this  paper,  detailed  discussions  of 
measures  of  effectiveness  and  analytical  methods 
to  be  used  in  evaluating  the  criterion  will  be 
deferred  to  subsequent  reports. 

METHODOLOGY 

WARC  undertook  three  research  tasks  to  support 
development  of  a  criterion  for  evaluating  the  test: 

•  A  literature  review  of  advertising  and 
marketing  research  relevant  to  evaluating 
advertising  effectiveness 

•  Primary  exploratory  research  in  the  form 
of  semistructured  interviews  aimed  at 


identifying  and  categorizing  prevailing 
motivations  for  collaborative  industry 
advertising 

•  Mathematical  models  of  the  effect  of 
advertising  on  military  enlistments 

After  a  brief  introduction  in  Chapter  1,  separate 
discussions  of  the  three  research  tasks  occupy 
Chapters  2,  3,  and  4  of  the  report.  The  final 
chapter  provides  conclusions  based  on  the 
research  efforts  and  presents  a  test  evaluation 
criterion. 

FINDINGS  OF  THE  LITERATURE 
REVIEW 

Although  several  studies  have  focused  on 
generic  or  commodity  advertising  for 
agricultural  products,  the  overwhelming 
majority  of  advertising  research  has  been 
conducted  at  the  firm  or  brand  level. 
There  are  two  broad  lines  of  this  research:  One 
attempting  to  relate  aggregate  phenomena  such  as 
firm  or  brand  sales  to  advertising  resources  and  a 
second  investigating  the  impact  of  advertising 
resources  on  consumer  attributes  such  as 
awareness  of  advertising,  copy  recall,  and 
intention  to  buy. 

Considerable  support  exists  in  the 
literature  for  aggregate  sales  response 
models.  These  models  have  also  been  applied  to 
the  Service-specific  military  enlistment 
environment  with  some  success. 


Although  there  is  no  clear  consensus  on 
the  role  of  consumer  attribute  variables, 
there  is  some  significant  evidence  that 
stated  purchase  intention  measures  are 
valid  predictors  of  purchase  behavior. 
The  literature  review  reinforces  the  requirement 
that  any  intermediate  variables  employed  in 
evaluating  the  test  must  be  rigorously  tested  for 
both  validity  and  reliability. 

No  guidance  is  provided  in  the  literature 
concerning  the  simultaneous  evaluation 
of  industry  advertising  and  firm-specific 
advertising.  Several  studies  have  estimated  the 
effect  of  advertising  on  sales  for  a  number  of 
firms  in  the  same  industry.  However,  results  of 
these  investigations  have  differed  sharply,  and 
none  of  these  analyses  considered  a  collaborative 
or  jointly  sponsored  advertising  campaign.  Other 
studies  have  estimated  the  effects  of  advertising 
on  sales  for  generic  or  commodity  advertising 
campaigns  but  haye  not  simultaneously  consi¬ 
dered  firm-specific  campaigns. 

A  distinction  between  national,  corporate 
advertising  and  cooperative  (largely  trade 
channel)  industry  advertising  was 

identified.  In  a  recent  book  on  this  topic, 
Young  and  Greyser  (1983)  assert  that: 

"There  exists  very  little  in  the  way  of  formal 
evaluation  methods  for  assessing  co-op's 
effectiveness.  The  usual  advertising  tracking 
services  and  evaluation  services  offer  no 
systematic  way  of  helping  managers  assess  co¬ 
op's  effectiveness." 


Previous  studies  of  military  recruiting 
advertising  are  of  limited  usefulness. 
The  focus  of  the  effort  to  understand  the  effects  of 
military  recruiting  advertising  has  been  primarily 
at  the  individual  Service  level.  Although  a 
substantial  number  of  enlistment  supply  models 
have  been  developed  since  the  inception  of  the  All 
Volunteer  Force  in  1973,  very  few  of  these 
studies  estimate  the  effect  of  advertising  on 
enlistments.  Those  models  which  do  make  these 
estimates  disagree  about  the  underlying  model 
structure  and  about  the  advertising  effect  on 
enlistments.  Despite  the  lack  of  consensus  about 
the  magnitude  of  advertising  effects,  the  literature 
on  military  advertising  indicates  that  marketing 
variables  do  have  significant  effects.  The  fielding 
of  a  controlled  experiment  such  as  the  FY  1984 
Advertising  Mix  Test  is  advocated  in  this 
literature. 

The  underlying  themes  in  the  evaluation 
literature  are  sales  effectiveness  and 
economic  efficiency.  In  the  private  sector, 
these  competing  factors  arc  incorporated  into 
profit  maximization  models  where  advertising 
budgets  are  set  so  that  the  incremental 
contribution  from  the  last  sale  just  equals  the 
advertising  cost  of  achieving  the  sale. 
Unfortunately,  since  a  mechanism  for  measuring 
recruiting  "sales"  in  terms  of  dollars  is  lacking, 
the  concept  of  profit  maximization  cannot  be 
applied  directly  to  evaluating  military  recruiting 
advertising  policies. 


FINDINGS  OF  INDUSTRY  INTER*  youths  approaching  and  in  the  prime 

VIEWS  enlistable  age  and  among  their  influences. 

Although  collaborative  advertising  is  not  rare  in 


the  marketplace,  the  literature  dealing  with  its 
evaluation  offers  little  guidance  for  evaluating  the 
FY  1984  Advertising  Mix  Test.  To  gain  a  better 
understanding  of  motivations  for  collaborative 
advertising,  WARC  project  staff  interviewed  20 
industry  trade  associations.  Five  major  moti¬ 
vations  for  collaborative  advertising  were 
identified: 

•  To  achieve  economies  of  scale 

•  To  address  secondary  target  groups 

•  To  address  primary  target  groups  in  a 
different  way 

•  To  eliminate  "free  riders" 

•  To  stimulate  goodwill 

These  objectives  fall  naturally  into  the  efficiency/ 
effectiveness  dichotomy  familiar  from  the 
literature  review  and  translate  fairly  directly  into 
hypotheses  about  the  roles  of  Joint  and  Service- 
specific  advertising. 

FINDINGS  OF  CONCEPTUAL  MODELS 
Four  conceptual  models  were  developed  to  aid  in 
understanding  advertising  contributions  to  the 
enlistment  process.  The  models  hypothesize 
two  kinds  of  contributions  from  military 
recruiting  advertising: 

•  Direct  contributions  to  closing  enlistment 
contracts 

•  Indirect  contributions  to  improved  atti¬ 
tudes  toward  military  service  among 


The  conceptual  models  minimize  the  sum  of 
Service-specific  and  Joint  advertising  expendi¬ 
tures  (efficiency)  while  assuring  sufficient  adver¬ 
tising  expenditures  that  each  Service  meets  its 
enlistment  contract  requirements  (effectiveness). 

In  addition  to  the  recurring  themes  of 
efficiency  and  effectiveness,  several 
unique  elements  of  the  recruiting 
environment  were  highlighted  in  the 
formulation  and  evaluation  of  these 
conceptual  models: 

•  The  minimal  impact  of  repeat  purchases 

•  A  strictly  sequential  buying  cycle 

•  An  intermediate  stated  purchase  intention 
measure  with  significant  predictive  vali¬ 
dity 

•  Policy  constraints  prescribing  quotas  and 
quality  standards 

•  A  narrow  and  transitory  primary  target 
group 

•  Impact  of  influences 

•  Considerable  brand  switching  by  appli¬ 
cants 

CONCLUSION 

The  literature  review,  the  survey  of 
industry  advertisers  and  the  conceptual 
model  development  each  support  the  use 
of  both  efficiency  and  effectiveness 
criteria  for  evaluating  the  FY  1984 
Advertising  Mix  Test. 


Strong  support  was  found  for  using 
aggregate  sales  reponse  (or  the  responses 
of  Intermediate  variables  which  validly 
predict  sales)  as  an  effectiveness  cri¬ 
terion.  Short-term  sales  response  contributions 
include  meeting  accession  missions,  meeting  con¬ 
tract  missions  and  maintaining  quality  standards. 
Longer-term  enlistment  contributions  of  adver¬ 
tising  include  maintaining  favorable  attitudes 
toward  military  service  generally  and  toward 
individual  branches  as  well  as  promoting  favor¬ 
able  behavior  among  high-quality  youth. 

Strong  support  was  also  found  for  using 
an  economic  efficiency  criterion.  In  the 
private  sector,  economic  efficiency  is  typically 


conceptualized  as  profit  maximization.  Because 
recruiting  "sales"  cannot  presently  be  translated 
into  dollars,  profit  maximization  is  replaced  by 
cost  minimization  as  an  economic  efficiency 
criterion  for  evaluating  the  test. 

Consequently  the  following  criterion  will  be  used 
to  evaluate  the  advertising  budget  policies  being 
tested  in  the  FY  1984  Advertising  Mix  Test: 

EVALUATION  CRITERION 
The  recommended  advertising  policy  will  be  one 
providing  the  necessary  short-term  and  long-term 
contributions  to  the  attainment  of  the  Services’ 
enlistment  requirements  at  minimum  advertising 
cost  to  the  Department  of  Defense. 


CHAPTER  1 
INTRODUCTION 


The  FY  1984  Advertising  Mix  Test  is  one  of 
the  most  ambitious  and  complex  research  initia¬ 
tives  undertaken  in  the  area  of  military  recruiting. 
Further,  it  is  likely  the  most  intricate  and  ambi¬ 
tious  of  marketing  field  experiments  conducted  to 
date.  The  objective  of  the  experiment  is  to 
evaluate  the  effectiveness  of  enlisted,  active-duty, 
non-prior-service  (NPS)  military  recruiting  adver¬ 
tising.  As  a  result,  the  test  is  designed  to  provide 
a  solid  quantitative  and  analytic  basis  for  estab¬ 
lishing  the  funding  level  for  such  advertising  by 
the  military  and  for  the  division  of  such  adver¬ 
tising  funding  between  the  Joint  Recruiting 
Advertising  Program  (JRAP)  and  the  individual 
Services. 

The  experiment  was  initiated  because  "the 
Department  of  Defense  does  not  have  a 
methodology  which  relates  and  quantifies  the 
effect  of  varying  levels  of  advertising  to  actual 
enlistment"1  The  experiment  has  been  designed 
as  "an  in-market  test  which  would  generate 
quantitative  data  to  help  answer  the  question: 
"What  is  the  optimum  mix  of  Joint/Service- 
specific  recruitment  advertising  for  achieving 
active,  enlisted,  NPS  goals  at  different  levels  of 
total  DoD  recruitment  advertising?"'2 

Given  the  objectives  and  motivations  for  the 
test,  this  report  is  concerned  with  recommending 
the  basis  on  which  alternative  advertising  policies 
may  be  evaluated. 


Evaluating  the  effectiveness  of  advertising  is  a 
complex  issue.  Military  recruiting  advertising 
occurs  at  different  levels  of  execution,  as  does 
advertising  in  many  other  environments.  At  one 
level,  specific  products,  services  and  programs 
are  advertised.  Examples  include  the  Army 
College  Fund,  the  Delayed  Entry  Program,  and 
the  Navy  Nuclear  Program.  Additionally,  each  of 
the  Services  advertises  for  reserves,  officers,  and 
prior-service  personnel.  At  another  level,  the 
Services  advertise  to  differentiate  themselves  from 
one  another  and  from  other  employment  or 
training  opportunities.  Finally,  the  Department  of 
Defense  collaborates  in  advertising  through  the 
Joint  Recruiting  Advertising  Program. 

This  same  variety  of  levels  of  advertising 
prevails  in  many  private  sector  organizations.  For 
example,  financial  institutions  such  as  commercial 
banks  or  Savings  and  Loan  Associations 
frequently  advertise  specific  products  or  programs 
such  as  checking  or  NOW  accounts,  individual 
retirement  accounts,  etc.  At  another  level,  they 
attempt  to  differentiate  the  institution  itself  by 
advertising  such  features  as  reliable  service, 


1  Memorandum  dated  8  July  1983  from 
Dr.  Lawrence  J.  Korb,  Assistant  Secretary  of 
Defense  (Manpower,  Reserve  Affairs  & 
Logistics)  to  the  Assistant  Secretaries  of  the 
Army,  Navy,  and  Air  Force. 

2Korb,  8  July  1983. 


convenient  hours  and  locations,  friendly  or  fast 
response.  Finally,  these  institutions  collaborate  to 
advertise  the  more  generic  benefits  of  the 
"industry"  of  Savings  and  Loans  Associations  or 
Full-Service  Banks. 

Since  tins  test  is  the  first  systematic  research 
effort  aimed  at  investigating  advertising  effective¬ 
ness  across  different  levels  of  advertising  execu¬ 
tion,  no  generally  accepted  evaluation  criterion  is 
available.  Consequently,  three  research  initiatives 
were  undertaken: 

A  literature  review  of  advertising  and 
marketing  research  relevant  to  eval¬ 
uating  advertising  effectiveness.  The 
review  focused  on  firm  or  brand  advertising, 
industry  advertising,  and  advertising  for 
military  enlistments. 

Primary  exploratory  research  in  the 
form  of  semi-structured  interviews 
with  industry  trade  associations  aimed 
at  identifying  and  categorizing  pre¬ 


vailing  motives  for  collaborative 
industry  advertising. 

Mathematical  models  of  advertising's 
effect  on  military  enlistments.  These 
models  were  conceptual  mathematical 
representations  of  Service-specific 
and  Joint  advertising  effects  devel¬ 
oped  from  a  DoD  perspective. 

The  results  of  these  separate  research 
initiatives  are  discussed  in  the  next  three  chapters 
of  this  report  The  final  chapter  of  this  report 
presents  conclusions  based  on  the  three  research 
tasks  and  describes  the  recommended  criterion  for 
evaluating  the  FY  1984  Advertising  Mix  Test 
The  final  chapter  also  briefly  describes  some 
candidate  evaluation  measures  and  analytical 
methods.  This  description  of  the  candidate  mea¬ 
sures  and  analytical  methods  is  preliminary  and 
subject  to  change.  A  detailed  discussion  of  mea¬ 
sures  of  advertising  effectiveness  and  analytical 
methods  will  be  the  subject  of  a  separate  report 


CHAPTER  2 

A  LITERATURE  REVIEW 


INTRODUCTION 

This  chapter  presents  a  review  of  the 
advertising  and  marketing  research  literature 
relevant  to  evaluating  advertising  effectiveness. 
The  review  is  organized  into  three  sections 
covering  commercial  advertising  at  the  brand  or 
firm  level  of  analysis,  commercial  advertising  at 
the  industry  or  multi-firm  level  of  analysis,  and 
advertising  for  military  enlistments.  A  final 
section  summarizes  the  relevance  of  this  literature 
to  developing  a  criterion  for  evaluating  the  FY 
1984  Advertising  Mix  Test. 

COMMERCIAL  ADVERTISING  FROM  A 
BRAND  OR  FIRM  PERSPECTIVE 

The  overwhelming  majority  of  advertising 
research  in  the  commercial  sector  has  been 
conducted  at  the  firm  or  brand  level  of  advertising 
execution.  A  large  body  of  this  research  has 
investigated  the  sales  effects  of  advertising. 

One  of  the  earliest  and  most  widely  cited 
aggregate  response  models  is  presented  by  Vidale 
and  Wolfe  (1957).  After  presenting  the  results  of 
many  empirical  studies,  they  formulated  a 
dynamic  model  of  sales  as  a  function  of  three 
components:  sales  decay,  a  saturation  effect,  and 
an  advertising  response  coefficient.  Nerlove  and 
Arrow  (1962)  had  a  different  conceptualization 
for  the  effects  of  advertising.  Their  model 
considered  advertising  to  affect  the  accumulation 
of  "goodwill,"  and  the  goodwill  in  turn  to  affect 
sales. 


Three  comprehensive  review  articles  cover 
more  recent  work  assessing  the  aggregate  sales 
effects  of  advertising.  They  examine  a  large 
number  of  field  experiments  and  analyses  of 
historical  data.  One  article.  Little  (1979),  focuses 
on  the  aggregate  modeling  of  the  advertising/sales 
relationship.  The  others,  Aaker  and  Carman 
(1982),  and  Simon  and  Arndt  (1980),  focus  on 
the  implication  of  this  body  of  research  for  the 
practitioner.  The  relevant  elements  of  these 
articles  are  discussed  below. 

At  the  level  of  firm-  or  brand-specific 
advertising,  Little  (1979)  concludes  that  several 
phenomena  have  empirical  support  and  should  be 
considered  in  building  aggregate  models  of 
advertising  responses: 

•  Sales  respond  dynamically  upward 
and  downward  to  increases  and 
decreases  in  advertising  and 
frequently  do  so  at  different  rates. 
Thus,  advertising/sales  relationships 
frequently  involve  cumulative  or  lag 
effects.  The  lag  effects  observed  have 
been  different  for  increases  in  advertising 
than  for  decreases. 

•  The  steady-state  response  of  sales 
to  advertising  can  follow  a  concave 
or  S-shaped  curve.  The  sales 
response  to  advertising  exhibits 
diminishing  returns  and  may  exhibit  a 


threshold  effect  as  well.  There  are  often 
positive  sales  at  zero  advertising. 

•  Competitive  advertising  affects 
sales  at  the  firm  or  brand  level. 
Hence,  firm  or  brand  sales  often  depend 
not  only  on  the  advertising  level  and 
policy  of  the  firm  or  brand  but  also  on  the 
advertising  level  of  competitors. 

•  The  dollar  effectiveness  of  adver¬ 
tising  can  change  over  time  as  a 
result  of  changes  in  media,  copy, 
and  other  factors. 

•  Products  sometimes  respond  to 
increased  advertising  with  a  sales 
increase  that  falls  off  even  as 
advertising  is  held  constant.  Hence, 
the  effects  of  an  increased  level  of 
advertising  can  decay  even  when  the 
increased  level  of  advertising  is 
maintained  over  time. 

Aaker  and  Carman  (1982)  provide  a  useful 
taxonomy  by  which  the  impact  of  advertising  may 
be  classified  into  five  specific  aspects  of  the 
advertising/sales  relationship.  This  classification 
scheme  is  useful  since  there  has  been  considerable 
confusion  in  the  literature  about  the  terminology 
used  in  discussing  the  advertising/sales  relation¬ 
ship.  The  specific  aspects  of  the  advertising/sales 
relationship  discussed  by  them  are  as  follows: 

•  The  short-run  brand  demand  adver¬ 
tising  effect.  This  effect  is  the  brand 
sales  generated  in  a  time  period  which 


were  caused  by  advertising  run  during  the 
same  period. 

•  The  short-run  carryover  effect. 
This  effect  is  the  brand  sales  generated 
during  the  subsequent  few  time  periods 
after  the  advertising  has  appeared. 

•  The  primary  demand  advertising 
effect.  This  is  the  effect  of  advertising  in 
generating  overall  category  or  industry 
sales  (rather  than  changing  market 
shares). 

•  The  response  of  competitive  mar¬ 
keting  to  an  increase  or  decrease  in 
brand  advertising.  Changing  the 
brand  or  firm  advertising  may  cause  brand 
or  firm  competitors  to  change  their 
advertising  or  marketing  expenditures. 

•  The  long-run  advertising  impact  on 
the  process  of  goodwill  creation, 
persistence  and  decay.  This  effect 
may  involve  many  purchase  cycles  and 
could  involve  primary  demand  expansion. 
The  major  long-term  effect  for  most 
established  products  is  to  affect  a  brand's 
"goodwill"  or  the  loyalty  it  enjoys  among 
its  customers. 

Military  Services'  recruiting  advertising  may 
be  expected  to  affect  enlistments  in  some  or  all  of 
these  ways.  The  competitive  effect  and  goodwill 
effect  may  be  reduced  substantially  in  the  military 
environment.  The  competitive  effect  is  reduced  in 
the  short-term  by  the  military  advertising 


budgeting  process.  The  goodwill  effect  is 
reduced  by  the  one-time  nature  of  the  purchase 
decision.  Other  long-term  effects  of  advertising 
on  enlistments  are  quite  possible  in  the  military 
recruiting  area  and  could  include,  for  example,  the 
effects  of  advertising  on  the  population  not  yet  old 
enough  to  enlist 

Recently,  aggregate  response  models  have 
incorporated  advertising  effects  into  new  product 
diffusion  models.  These  models  represent  the 
"level  of  spread  of  an  innovation  among  a  given 
set  of  prospective  adopters  in  terms  of  a  simple 
mathematical  function  of  time  that  has  elapsed 
since  the  introduction  of  the  innovation"  (Mahajan 
and  Muller,  1979).  Generally,  diffusion  models 
track  the  flow  of  customers  across  three  distinct 
market  segments:  people  who  are  not  aware  of 
the  product,  people  who  are  aware  of  but  have  not 
bought  the  product,  and  those  who  have  made  a 
purchase.  The  concepts  of  external  and  internal 
influence  on  the  potential  adopter  are  important  in 
these  models.  Mahajan  and  Muller  (1979)  define 
external  influence  as  direct  influence  on  the 
purchase  behavior  of  an  individual  through 
marketing,  promotional  material,  or  personal 
discussion  with  professional  sales  people. 
Internal  influence,  on  the  other  hand,  results  from 
social  interaction  and  is  that  effect  exerted  by 
members  of  a  social  system  on  each  other  (e.g., 
word  of  mouth).  Although  there  have  been 
various  extensions  to  these  models  (as  best 
summarized  and  reviewed  by  Mahajan  and  Muller 
(1979),  and  Engel,  Kollat  and  Blackwell  (1978)), 
these  authors  admit  that  their  usefulness  is  limited 
by  a  distinct  lack  of  empirical  validation  and 
testing.  While  diffusion  model  constructs  are  not 


directly  useful  to  the  FY  1984  Advertising  Mix 
Test,  the  underlying  behavioral  processes  (i.e., 
personal  influences)  warrant  further  attention. 
Preliminary  research  using  longitudinal  follow-up 
data  for  Youth  Attitude  Tracking  Study  respon¬ 
dents  is  promising. 

These  aggregate  sales  response  models  use 
the  estimated  sales  effect  of  advertising  as  a  basis 
for  evaluating  advertising  effectiveness.  Econo¬ 
mic  criteria  are  used  to  choose  between 
alternatives  deemed  adequately  effective.  For 
example,  profit  maximizing  formulations  set 
advertising  budgets  so  that  the  incremental 
revenue  from  the  last  additional  sale  is  just  equal 
to  the  additional  costs  (including  advertising) 
needed  to  achieve  these  sales  (i.e.,  the  advertising 
budget  is  increased  until  marginal  profit  is  zero). 
Budgets  are  set  for  competing  marketing  alterna¬ 
tives  using  similar  criteria. 

An  additional  stream  of  brand  or  firm 
advertising  research  has  been  conceptualized  at 
the  consumer  or  micro-level.  One  of  the  earliest 
formal  models  of  the  entire  purchase  process  was 
proposed  by  Nicosia  (1966).  Nicosia  viewed  the 
purchase  process  as  consisting  of  three  compo¬ 
nents  (motivation,  attitude,  purchase)  which  he 
expressed  quantitatively  in  the  form  of  a  system 
of  differential  equations.  This  model  was  limited, 
however,  since  he  only  included  consumer 
motivations  and  attitudes  as  the  relevant  variables. 
Extensions  to  this  model  include  such  considera¬ 
tions  as  memory  components,  information  search 
elements,  external  influence  and  individual  charac¬ 
teristics  (Katz  and  Lazarsfeld  (1955),  McGuire 
(1976),  Newell  and  Simon  (1972)). 
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More  recently,  Bettman  (1979)  has  proposed 
yet  another  model  based  explicitly  upon  informa¬ 
tion  processing  theory.  He  views  die  consumer 
as  an  active  seeker  and  evaluator  of  information. 
That  is,  the  consumer  is  characterized  as  having  a 
set  of  goals  from  which  point  he/she  interacts 
with  his  or  her  environment,  seeking  information, 
processing  that  information,  and  then  selecting 
among  a  set  of  alternatives.  The  basic  compo¬ 
nents  of  Bettman’s  choice  process  include  process¬ 
ing  capacity,  motivation,  attention  and  perception, 
information  acquisition  and  evaluation,  memory, 
decision  processes  and  learning.  Choice  provides 
the  focal  point  of  the  theory,  and  the  emphasis  is 
not  simply  on  outcomes  but  on  the  processes 
underlying  the  choice  decision. 

Despite  what  appear  to  be  very  different 
model  structures,  all  of  these  theories  have  certain 
elements  in  common  (Lunn,  1974): 

•  A  focus  on  choice  as  a  process,  rather 
than  choice  just  as  the  purchasing  act  itself 

•  A  view  of  choice  as  purposeful  behavior, 
with  the  consumer  being  an  active  infor¬ 
mation  seeker  and  user  of  both  internal 
and  external  information 

•  A  belief  that  behavior  is  caused  and  thus 
can  be  explained 

•  A  belief  that  consumers  limit  the  amount 
of  information  processed 

•  The  idea  that  feedback  based  on  outcomes 
from  choices  can  affect  later  decisions 


A  number  of  researchers  including  Strong 
(1925),  Rogers  (1982),  and  Lavidge  and  Stiener 
(1961)  have  described  and  extended  the  basic 
"Hierarchy  of  Effects"  model.  Generally,  this 
model  proposes  three  major  stages  in  the  con¬ 
sumer  purchase  process.  The  various  iterations 
of  this  model  are  based  on  the  assumption  that 
knowledge  precedes  attitudinal  change,  which  pre¬ 
cedes  behavioral  change.  However,  these  models 
have  been  criticized.  Palda  (1966)  presents 
empirical  evidence  which  contradicts  the  Lavidge 
and  Stiener  model.  Others  have  questioned  these 
models  on  two  grounds:  (1)  all  individuals  do  not 
necessarily  move  through  each  successive  stage; 
and  (2)  it  is  inconclusive  that  attitude  change  must 
precede  behavioral  change.  Indeed  some  evi¬ 
dence  attests  to  the  reverse  notion  that  attitude 
change  follows  behavioral  change  (Kiesler, 
Collins,  and  Miller  (1969);  Krugman  (1965)). 

The  critical  problem  with  the  consumer 
behavior  models,  however,  is  the  difficulty  of 
empirically  testing  and  evaluating  them  for 
validity  and  reliability.  As  a  result,  some 
marketing  researchers  have  recently  turned  to 
stochastic  models  of  behavior  (e.g.,  Massey, 
Montgomery,  and  Morrison  (1970)).  Recent 
publications  include  those  of  Bass  (1974), 
Kalwani  and  Morrison  (1977),  and  Lehman 
(1976).  All  of  these  models  make  the  assumption 
that  consumer  behavior  has  a  large  random 
component  which  is  in  principle  unpredictable. 
These  models  are  generally  concerned  with 
predicting  purchase  behavior  through  the  use  of 
probability  density  functions  of  individual 
purchase  probabilities  and  aggregate  switching 
matrices  associated  with  brand  loyalty.  Although 


data  limitations  continue  to  retard  their  usefulness, 
some  success  has  been  achieved  with  these 
models. 

When  applied  to  assessing  advertising 
effectiveness,  these  consumer  behavior  studies 
focus  on  "intermediate"  variables  such  as  aware¬ 
ness  of  advertising,  copy  recall,  attitudes  toward 
the  category  or  brand,  and  stated  purchase 
intention.  In  general,  no  clear  consensus  has 
emerged  on  the  role  of  intermediate  variables. 
Key  issues  in  this  area  are  predictive  validity  (the 
degree  to  which  a  change  in  an  intermediate 
variable  such  as  advertising  awareness  accurately 
predicts  a  change  in  a  behavior  of  interest  such  as 
sales  or  purchase  action),  instrument  or  measure¬ 
ment  validity  (the  degree  to  which  changes  in  the 
intermediate  variable  itself  can  be  validly 
assessed),  and  reliability  (the  degree  to  which 
effects  detected  among  one  group  or  sample  can 
be  generalized).  A  notable  exception  hoe  is  the 
use  of  purchase  intention  measures  to  predict 
purchase  behavior.  Kalwani  and  Silk  (1982) 
conclude  a  comprehensive  review  of  this  literature 
with  the  statement  that  "for  users  of  intentions 
data,  this  paper  offers  some  welcome  evidence 
that,  across  a  broad  range  of  conditions,  such 
measures  do  possess  a  statistically  significant 
degree  of  predictive  validity." 

With  the  exception  of  stated  purchase 
intentions,  then,  micro-level  consumer  behavior 
constructs  using  intermediate  variables  do  not 
seem  to  be  promising  approaches  to  evaluating  the 


the  test  or  any  constructs  which  rely  on  them  must 
be  rigorously  tested  for  both  validity  and  relia¬ 
bility. 

ADVERTISING  FROM  AN  INDUSTRY 
PERSPECTIVE 

At  the  level  of  industry  or  collaborative 
advertising,  limited  guidance  for  the  test  is  pro¬ 
vided  in  the  literature.  No  systematic  empirical 
evidence  is  available  for  the  simultaneous 
evaluation  of  collaborative  or  industry  advertising 
and  firm-specific  advertising.  However,  a  number 
of  studies  have  estimated  the  effect  of  advertising 
on  sales  for  firms  in  the  same  industry. 

These  efforts  have  included  work  by  Lambin 
(1972)  on  gasoline,  Beckwith  (1972),  Sexton 
(1970),  and  Aaker,  Carman,  and  Jacobson  (1982) 
on  frequently  purchased  consumer  goods,  Simon 
(1969)  on  liquor  and  Buzzell  and  Baker  ( 1972)  on 
domestic  automobiles.  In  addition,  a  substantial 
amount  of  work  has  been  done  using  data  on 
cigarette  sales  and  advertising  for  the  major  tobac¬ 
co  companies.  These  efforts  have  included  Telser 
(1962),  Schnabel  (1972),  Schmallensee  (1972), 
Peles  (1971),  Bass  (1969),  Dominquez  and  Page 
(1971),  Rao  (1972)  and  Horsky  (1977).  Miles 
and  Snow  (1982)  qualitatively  evaluate  these  data. 

Results  of  these  investigations  have  differed 
sharply.  Simon  (1969)  reports  consistent  under- 
advertising  by  all  brands  in  one  industry,  while 
Aaker,  Carman,  and  Jacobson  (1982)  report  over¬ 
advertising  by  all  brands  in  another  industry. 


FY  1984  Advertising  Mix  Test.  Furthermore, 
any  intermediate  variables  employed  in  evaluating 


None  of  these  analyses  considered  a  collaborative 
or  jointly  sponsored  advertising  campaign. 
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Otter  studies  have  estimated  the  effect  of 
advertising  on  sales  for  generic  industry  adver¬ 
tising  of  several  agricultural  commodities.  Since 
the  early  1970's,  the  Federal  Government  has 
provided  a  stimulus  for  such  advertising  in  the 
agricultural  sector  through  "Research  and 
Promotion"  enabling  legislation  for  various 
commodities.  Studies  of  the  sales  effect  of 
generic  commodity  advertising  include  works  by 
Ward  (1973,  1974,  1975,  1976),  Ward  and  Behr 
(1980),  and  Ward  and  Davis  (1978),  Hochman, 
Regev  and  Ward  (1974)  on  citrus  fruits  and 
Thompson  and  Eiler  (1975,  1977),  Clement, 
Henderson  and  Eley  (1965),  and  Thompson, 
Eiler  and  Forker  (1976)  on  fluid  milk  advertising. 
Generally,  these  efforts  use  aggregate  response 
models  to  assess  the  effect  of  generic  advertising 
on  returns  to  commodity  producers  and  on  indus¬ 
try  structure.  Some  assessments  also  consider 
industry  price  setting  and  regulatory  mechanisms 
(e.g.,  Thompson  and  Eiler  (1975,  1977),  Ward 
(1975)).  As  a  whole,  these  studies  successfully 
estimated  the  sales  effect  of  industry  advertising 
and  used  economic  theory  to  inform  both  advertis¬ 
ing  decisions  and  other  industry  policy  decisions. 
Importantly,  however,  none  of  these  efforts 
considered  firm-specific  advertising  efforts. 

A  limited  amount  of  normative  literature  exists 
regarding  collaborative  or  jointly  sponsored 
industry  advertising.  In  the  only  publication 
addressing  the  role  of  collaborative  industry 
advertising,  Hugh  Agnew  (1926)  presents  a 
historical  review  of  industry  advertising  which  he 
categorizes  as  actually  a  type  of  "cooperative" 
advertising.  Industry  advertising  is  that  in  which 
producers  of  products  in  the  same  industry 


promote  the  entire  industry  by  combined  efforts  in 
advertising.  Historically,  the  objectives  of 
industry  advertising  were  to  improve  market 
conditions,  to  combat  a  common  enemy,  to  pre¬ 
pare  background  for  the  campaigns  of  members 
firms  and  in  general  to  increase  sales. 

According  to  Agnew,  early  industry  adver¬ 
tising  efforts  addressed  several  different  specific 
purposes.  Campaigns  were  developed  to  educate 
the  consumer.  For  example,  the  National  Electric 
Light  Association  campaign  showed  the  diffi¬ 
culties  of  maintaining  electric  light  service  so  that 
patrons  would  understand  why  service  is  inter¬ 
rupted  by  storms  and  accidents.  Campaigns  were 
also  designed  to  defend  the  industry  from  outside 
attacks.  The  American  Face  Brick  Association 
campaign  directed  its  efforts  at  combating  the 
propaganda  that  brick  houses  were  more  damp, 
unsanitary,  and  expensive  than  houses  con¬ 
structed  with  other  building  materials.  Correction 
of  bad  trade  practices  was  another  rationale  for 
industry  advertising.  When  competition  led  to  the 
practice  of  guaranteeing  auto  tires  in  terms  of 
mileage,  the  Rubber  Association  of  American  saw 
this  as  a  bad  trade  practice  and  tried  to  correct  it 
by  showing  how  it  was  actually  the  care  of  tires 
that  made  them  last  and  that  a  mileage  guarantee 
would  lead  to  consumer  neglect  Finally,  towards 
the  goals  of  expanding  sales,  the  "Say  it  With 
Flowers"  campaign  extended  the  practice  of 
sending  flowers  at  times  of  sorrow  to  sending 
flowers  on  festive  occasions  throughout  the  year. 

Agnew  defines  two  other  types  of  "coopera¬ 
tive"  advertising:  (1)  where  a  supplier  and 
distributor  jointly  collaborate  to  promote  a  product 
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or  products  in  which  they  are  mutually  interested 
(e.g.,  G.E.  collaborates  with  a  local  appliance 
store  in  promoting  G.E.  washing  machines);  and 
(2)  where  a  manufacturer  of  one  product 
recommends  its  products  specifically  for  use  with 
the  product  of  another  manufacturer.  This  latter 
type  of  cooperative  advertising  is  used  most  often 
in  the  marketing  plan  of  firms  making  goods 
which  consumers  purchase  frequently. 

In  a  recent  book.  Young  and  Greyser  (1983) 
discuss  cooperative  advertising.  Their  definition 
of  cooperative  advertising  is  limited  largely  to 
trade  channel  advertising  and  is  categorized  as: 

•  "Horizontal  cooperative  advertisingn 
which  refers  to  "advertising  sponsored  in 
common  by  a  group  of  retailers" 

•  "Ingredient-producer  cooperative  adver¬ 
tising"  which  is  "supported  by  raw 
materials  manufacturers" 

•  "Vertical  cooperative  advertising"  which  is 
"initiated  and  implemented  by  retailers  and 
partially  paid  for  by . . .  manufacturers" 

They  discuss  the  disadvantages  of  cooperative 
advertising  which  include  the  facts  that: 

(1)  sharing  the  costs  of  delivering  the  "message" 
often  results  in  conflicts  regarding  the  ad  content, 

(2)  the  flow  of  money  from  manufacturer  to 
retailer  is  reversed  from  its  normal  direction 
resulting  in  pressure  on  the  manufacturer  to 
maintain  the  goodwill  of  the  retailer,  and  (3)  dual 
signature  advertising  containing  both  a  message 
from  the  retailer  and  the  brand  message  from  the 


manufacturer  in  the  same  ad  results  in  a  more 
complicated  information-processing  challenge  for 
the  consumer. 

Given  these  potential  problems,  the  authors 
then  discuss  the  value  of  engaging  in  cooperative 
advertising  and  conclude  that  cooperative  adver¬ 
tising: 

•  Has  been  an  accepted  trade  practice  for 
years  and  may  be  viewed  as  routine 

•  Is  a  competitive  tool  within  the  trade  that 
appears  to  be  a  necessary  door-opener 
with  almost  all  large  retailers 

•  Serves  the  simple  function  of  telling  con¬ 
sumers  where  the  brand  is  available 

•  Results  in  increased  distribution  and  aids 
retailers  in  creating  immediate  sales  for  the 
company's  products 

•  Is  viewed  as  a  short-term  sales  stimulant 
since  consumers  can  be  susceptible  to 
effective  personal  selling  and  other  retail 
persuasion 

•  Can  be  effective  when  the  consumer  is 
closer  to  the  point  of  purchase 

Young  and  Greyser  explain  further  that 
national  advertising,  unlike  cooperative  adver¬ 
tising,  is  seen  as  building  the  image  of  both  the 
company  and  the  product  over  the  long  term. 
Since  there  is  a  perennial  conflict  about  how  much 
of  the  limited  advertising  resources  should  be 
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spent  on  national  versus  cooperative  advertising, 
Young  and  Greyser,  using  the  hierarchy  of  effects 
model,  argue  that  national  advertising  dollars 
should  be  spent  on  the  awareness,  knowledge  and 
liking  end  of  the  continuum  and  that  cooperative 
advertising  should  focus  on  the  preference  and 
action  end.  Since  military  enlistments  are  not 
distributed  through  large  retail  channels,  neither 
the  Joint  nor  the  Service-specific  advertising 
campaigns  are  cooperative  campaigns  in  the 
Young  and  Greyser  sense.  Nevertheless,  their 
hierarchy  of  effects  argument  results  in  at  least 
two  testable  hypotheses  for  the  FY  1984 
Advertising  Mix  Test:  (1)  Service-specific  adver¬ 
tising  is  more  effective  in  increasing  enlistments 
and  applicants,  and  (2)  Joint  advertising  is  more 
effective  in  improving  attitudes  and  awareness. 

With  respect  to  evaluation  criteria  per  se  for 
cooperative  advertising  these  authors  assert  that: 

"There  exists  very  little  in  the  way  of  formal 
evaluation  methods  for  assessing  co-op's 
effectiveness.  The  usual  advertising  tracking 
services  and  evaluation  services  offer  no 
systematic  way  of  helping  managers  assess  co¬ 
op's  effectiveness." 

Robert  Miles  (1982)  discusses  at  length  the 
strategies  used  by  the  Tobacco  Institute  (the  trade 
association  founded  by  the  "Big  Six"  tobacco 
manufacturers  in  1958)  in  direct  response  to  the 
mounting  controversy  over  smoking  and  health. 
Prior  to  this  time,  as  Lester  Telser's  study  (1962) 
confirms,  there  was  substantial  competition  in  the 
form  of  advertising  among  the  tobacco 
companies1  despite  Telser's  conclusion  that  their 
levels  of  advertising  were  high  enough  to  place 


them  at  the  point  where  they  were  experiencing 
diminishing  returns.  A  serious  threat  from  the 
external  environment  was  the  precipitating  factor 
that  drew  these  tobacco  rivals  together  for  the 
purpose  of  protecting  their  mutual  interests.  The 
concerted  efforts  of  the  Federal  Trade  Commis¬ 
sion  to  restrict  cigarette  advertising  and  label 
cigarette  products  with  health  warnings  provided 
the  impetus  for  the  Tobacco  Institute  to  initiate  an 
intensive  lobbying  effort  that  continued  through 
the  late  1960's.  Further,  Miles  discusses  exten¬ 
sively  the  forms  of  industry  collaboration  which 
evolved  from  1953  through  the  early  1970’s. 
However,  for  the  purposes  of  this  review,  a 
summary  of  the  conditions  and  factors  at  the 
individual  firm  level  which  help  explain  why 
collaborative  associations  emerge  or  decline  may 
be  more  relevant  His  observations  include  the 
following: 

•  Often  joint  ventures  are  formed  in 
anticipation  of  or  in  response  to  a  major 
external  stimulus. 

•  Legal  norms  prohibit  collaborative  efforts 
among  competitors  within  a  given 
industry  if  that  effort  would  tend  to  reduce 
competition  in  that  industry. 

•  Joint  ventures  tend  to  be  formed  more 
often  in  industries  characterized  by  a 
homogenous  population  of  firms.  "Where 

Reiser  notes  that  cigarettes  ranked  fourth  highest 
in  advertising  expenditures  across  all  categories 
listed  in  The  Source  Book  of  Income  (IRS). 


member  firms  compete  with  the  same  pro¬ 
duct  lines  for  similar  market  segments,  the 
relevance  and  potential  effect  of  external 
influences  and  threats,  as  well  as  oppor¬ 
tunities,  should  affect  them  similarly." 


•  Firms  will  take  an  industry  approach  if 
they  believe  they  can  achieve  greater 
economies  of  scale  and/or  greater  concen¬ 
tration  of  power  by  pooling  their 
resources  and  taking  a  united  stand  to  deal 
with  an  external  threat.  However,  the 
benefits  that  a  firm  gains  in  these  areas 
must  be  weighed  against  the  relative 
potential  loss  of  individual  organizational 
autonomy. 


MILITARY  RECRUITING  ADVER¬ 
TISING 

As  in  commercial  advertising  research,  studies 
of  military  recruiting  advertising  have  almost 
exclusively  been  limited  to  the  level  of  the 
individual  Service.  A  substantial  number  of  enlist¬ 
ment  supply  models  have  been  developed  since 
the  inception  of  the  All  Volunteer  Force  in  1973. 
Morey  and  McCann  (1983)  provide  a  review  of 
26  such  studies  conducted  since  1975.  Perelman 
(1983)  provides  another  such  review.  Generally, 
these  efforts  have  been  retrospective  analyses  of 
historical  data.  Further,  the  effect  of  advertising 
on  enlistrhents  is  estimated  in  relatively  few  of 
these  efforts.  Where  these  effects  are  estimated  or 
evaluated,  broad  disagreement  exists  about  the 
underlying  model  structures  and  about  the  effects 
of  advertising  on  enlistments. 


In  an  early  analysis  of  the  AVF,  Huck  (1974) 
conceptualized  an  enlistment  response  function  to 
advertisingexpenditureswhichexhibiteddiminish- 
ing  marginal  returns  and  provided  for  positive 
enlistments  at  zero  advertising  levels. 


Recent  empirical  studies  specifically  involving 
advertising  variables  include  Morey  and  McCann 
(1980)  who  examined  the  effects  of  national 
advertising  and  recruiter  levels  on  Navy  recruiting 
performance  using  monthly  cross-sectional  data 
for  1976  and  1977.  They  found  significant 
advertising  effects  (elasticities  of  0.19  for  total 
contracts,  and  0.12  for  high  school  graduate 
contracts)  and  recruiter  effects  (elasticities  of  0.44 
for  total  contracts  and  0.58  for  high  school 
graduate  contracts),  as  well  as  lagged  effects  for 
both.  They  also  reported  an  elasticity  of  0.16  for 
advertising  on  national  leads  generated.  Goldberg 
(1982)  analyzed  Navy  high  school  graduate 
enlistment  contracts  also  and  found  elasticities  of 
0.98  for  recruiters  and  0.25  for  advertising.  He 
noted  that  only  current  recruiter  levels  were 
important,  and  that  the  effects  of  advertising  were 
distributed  over  time.  Hanssens  and  Levien 
(1983)  discuss  another  econometric  study  of  the 
Navy  recruiting  market.  Their  econometric  model 
was  composed  of  three  equations-national  leads. 
Delayed  Entry  Program  (DEP)  contracts,  and 
immediate  shipment  contracts— which  were  esti¬ 
mated  individually.  Their  model  included  effects 
for  advertising  wear  out  (as  discussed  in  Simon 
(1982))  and  a  motivational  component  of  personal 
selling  (operationalized  as  a  function  of  quotas). 
Among  their  results,  a  strong  effect  of  advertising 
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on  national  leads  (elasticities  of  0.44)  was  found. 
In  addition,  they  found  a  significant  DEP  effect 
on  leads  (0.18)  and  quota  effect  on  DEP  contracts 
(-0.16).  They  also  found  significant  one  month 
lags  for  advertising. 

Carroll  and  Rao  (1981)  report  on  the  results 
of  a  field  experiment  conducted  for  the  Navy 
during  FY  1980.  They  report  significant  effects 
of  the  level  of  DoD  advertising  and  recruiters  on 
total  DoD  contracts,  indicating  that  these  variables 
had  an  effect  on  "primary  demand"  for  enlistment 
opportunities.  They  also  report  significant  effects 
on  Navy  local  advertising  and  Joint  advertising 
(which  was  not  a  treatment  component)  on 
different  categories  of  Navy  enlistment  contracts 
achieved.  Short-term  carry-over  effects  of  three 
to  four  months  for  some  advertising  elements 
were  evidenced. 

As  noted,  some  researchers  have  attempted  to 
estimate  the  effect  of  military  advertising  on  the 
intermediate  variable  of  leads.  As  Morey  and 
McCann  point  out,  however,  "the  only  lead  data 
bases  available  were  for  so-called  national  leads, 
the  result  of  national  advertising  campaigns 

- "  Until  better  lead  data  is  available  which 

includes  leads  from  local  advertising  sources  as 
well  as  self-initiated  lead  activity  (walk-ins  or  call- 
ins  to  the  recruiting  station),  considerable  poten¬ 
tial  measurement  error  impairs  this  approach. 
This  error  may  occur  because  substitution  across 
these  lead  categories  may  be  quite  substantial  as  is 
indicated  by  the  number  of  enlistees  who  do  not 
pass  through  the  intermediate  state  of  becoming 
national  "leads."  For  example,  such  national 
"leads"  account  for  less  than  25  percent  of  all 


reported  recruiter  contact  (Market  Facts  (1980), 
Carroll  etal  (1982)). 

In  summary,  the  review  of  the  literature  on 
military  advertising  reveals  that  marketing  vari¬ 
ables  have  significant  effects.  However,  disagree¬ 
ment  exists  on  the  magnitudes  of  these  effects, 
and  even  on  the  underlying  model  structures. 
Bass  (1981)  compares  and  discusses  the  metho¬ 
dological  weaknesses  in  the  models  of  Fernandez, 
Goldberg  and  Morey  and  McCann.  He  states  that 
"although  the  studies  do  provide  some  useful 
information,  the  policy  implications  should  be 
viewed  with  caution.  The  three  studies  should  be 
viewed  as  primitive  and  exploratory."  Bass  goes 
on  to  suggest,  however,  that  research  in  this  area 
shows  promise,  and  an  experimental  approach 
should  be  considered. 

In  concluding  their  review  article,  Morey  and 
McCann  make  several  recommendations  for 
needed  research: 

•  Integration  of  other  lead  data  into  enlist¬ 
ment  equations 

•  Incorporating  the  level  of  other  Services' 
marketing  efforts  into  enlistment  equation 
systems 

•  Assessing  the  impact  of  quotas  on 
enlistment  achievement 

•  Quantifications  of  uncertainty  in  decision 
making  models  by  specifying  confidence 
intervals  around  forecast  values 


Use  of  controlled  experiments 


These  suggestions  have  considerable  merit  in 
assessing  the  FY  1984  Advertising  Mix  Test 

SUMMARY  AND  CONCLUSIONS 

Considerable  support  for  aggregate  sales 
response  models  of  advertising  effectiveness 
exists  in  the  literature  at  the  brand-  or  firm-level  of 
analysis.  These  models  have  also  been  applied  in 
the  military  enlistment  environment  with  some 
success  amd  have  support  in  research  on 
commodity  advertising.  The  key  advertising 
phenomena  discussed  by  Little  (1979)  should  be 
considered  in  evaluating  advertising  response  for 
the  FY  1984  Advertising  Mix  Test 

Several  components  of  advertising's  effects 
on  sales  have  been  hypothesized  and  investigated 
in  these  aggregate  efforts.  Several  of  these 
components  are  most  relevant  to  military  enlist¬ 
ment  advertising  and  to  the  FY  1984  Advertising 
Mix  test  Specifically,  the  short-run  brand 
demand  effect,  the  short-run  carry-over  effect, 
and  the  primary  demand  effect  are  most  salient  to 
this  effort  The  competitive  response  effect  is 
attenuated  in  the  military  enlistment  arena  by  the 
military  advertising  budgeting  process.  The  long- 
run  effects  of  advertising  on  goodwill  concep¬ 
tualized  as  repeat  purchase  are  of  little  relevance  to 
this  study.  However,  the  narrow  and  transitory 


nature  of  the  primary  target  group  for  military 
recruiting  warrants  investigation  of  other  long¬ 
term  effects  such  as  the  development  of  positive 
attitudes  among  individuals  who  are  entering  and 
leaving  the  group. 

The  literature  provides  support  for  the  use  of 
stated  purchase  intention  measures  as  statistically 
valid  predictors  of  sales.  Recent  methodological 
advances  (see,  for  example,  Morrison  (1979))  in 
this  area  are  encouraging.  Orvis  (1982)  provides 
direct  support  for  the  use  of  such  a  measure  in  the 
military  recruiting  area.  He  has  demonstrated  the 
predictive  validity  of  a  series  of  "propensity" 
measures  elicited  from  respondents  to  the  Youth 
Attitude  Tracking  Study. 

The  literature  provides  limited  guidance  for 
evaluating  the  effectiveness  of  industry 
advertising.  No  empirical  evidence  is  presented 
supporting  the  development  of  a  criterion  for 
evaluating  collaborative  industry  advertising 
campaigns  simultaneously  with  evaluating  firm- 
specific  advertising  effectiveness.  Also,  relatively 
little  relevant  normative  theory  is  available  for  this 
purpose.  A  key  issue  then  is  the  extent  to  which 
collaborative  advertising  may  be  reasonably 
evaluated  by  a  criterion  compatible  with  that  used 
for  brand-  or  firm-specific  advertising.  This  issue 
is  discussed  in  the  next  chapter  of  this  report. 
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CHAPTER  3 

MOTIVATIONS  FOR  COLLABORATIVE  INDUSTRY  ADVERTISING 


Why  do  organizations  decide  to  collaborate  on 
advertising?  What  are  the  objectives  of  collabora¬ 
tive  industry  advertising?  Do  these  objectives 
differ  in  kind  or  degree  from  the  objectives  of 
firm-  or  brand-specific  advertising?  Can  collabo¬ 
rative  advertising  be  expected  to  operate  in  a 
different  way  than  firm-  or  brand-specific 
advertising?  Little  theoretical,  normative,  or 
empirical  support  is  available  in  the  literature  to 
answer  these  and  similar  questions. 

However,  these  questions  are  important  to  the 


This  chapter  reviews  the  methodology 
employed  and  the  sample  interviewed  in  this 
exploratory  effort,  describes  the  findings 
regarding  the  motives  or  objectives  for  collabora¬ 
tive  industry  advertising,  and  discussess  the 
relevance  of  the  findings  to  the  evaluation  of  the 
FY  1984  Advertising  Mix  Test. 

METHODOLOGY 

A  sample  of  20  industry  trade  associations 
was  identified  as  the  subject  group  for  this 
research  task.  The  associations  selected  were 


evaluation  of  the  FY  1984  Advertising  Mix  Test. 
Since  1978,  the  Office  of  the  Assistant  Secretary 
of  Defense  and  the  military  Services  have  collabo¬ 
rated  in  enlistment  recruiting  advertising  through 
the  Joint  Recruiting  Advertising  Program.  A 
central  issue  in  the  FY  1984  Advertising  Mix  Test 
is  the  assessment  of  the  appropriate  mix  of  Joint 
and  Service-specific  advertising.  Gearly,  any 
meaningful  evaluation  of  the  test  must  incorporate 
any  differences  in  objectives  or  operations 
between  Joint  and  Service-specific  advertising. 


Since  little  guidance  is  available  in  the  litera¬ 
ture  to  identify  any  such  differences,  the  Wharton 
project  staff  conducted  primary  exploratory 
research  aimed  at  identifying  and  categorizing  pre¬ 
vailing  motives  for  collaborative  industry  adver¬ 
tising.  This  exploratory  research  was  conducted 
in  the  form  of  semi-structured  interviews  with 
industry  trade  associations  which  have  recently 
conducted  collaborative  advertising  campaigns. 


chosen  using  the  following  guidelines:  (1)  multi¬ 
million  dollar  advertising  budgets;  (2)  a  strong 
representation  by  industries  providing  financial 
services  (because  of  their  comparability  with  each 
other);  (3)  industries  competing  with  other 
industries  for  market  share  (e.g.,  the  beef  or  the 
pork  industry);  and  (4)  a  diverse  mixture  of 
industries. 


The  project  team  subsequently  excluded  one 
of  the  20  candidate  organizations  from  the  sample 
because  its  advertising  campaign  was  not  collabo- 
ratively  funded.  The  remaining  19  organizations 
(Table  3.1)  willingly  participated  in  the  research. 


Cognizant  representatives  from  each  organi¬ 
zation  were  interviewed  by  the  Wharton  project 
staff.  These  representatives  were  Directors  of 
Advertising,  Directors  of  Marketing,  or  Executive 
Directors  of  the  organizations  as  deemed 
appropriate  by  the  responding  organization.  In 
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Table  3.1 

COLLABORATIVE  ADVERTISING  SURVEY  PARTICIPANTS 


Florida  Department  of  Citrus 
National  Pork  Producers  Council 
American  Dairy  Association 
Florida  Fruit  and  Vegetable  Association 
National  Live  Stock  and  Meat  Board 
American  Florists  Marketing  Council 
Quality  Bakers  of  America 
Investment  Company  Institute 
Communication  Workers  of  America 
U.S.  Committee  for  Energy  Awareness 
The  American  Bankers  Association 
American  Council  of  Life  Insurance 
International  Ladies'  Garment  Workers'  Union 
Savings  and  Loan  Foundation,  Inc. 
Independent  Insurance  Agents  of  America 
American  Sheep  Products  Council,  Inc. 
American  Gas  Association 
American  Heart  Association 
Linen  Supply  Association  of  America 


one  case  the  commercial  advertising  agency  was 
viewed  as  the  most  knowledgeable  source  for 
responding  to  interview  questions. 

Fairly  indepth,  semi-structured  interviews 
were  conducted  with  these  representatives. 
Because  of  the  exploratory  nature  of  this  research, 
interview  lengths  were  permitted  to  vary  and 
ranged  from  30  minutes  to  more  than  90  minutes. 
An  initial  set  of  four  interviews  was  conducted  by 
personal  interview.  These  interviews  were  used 


to  refine  the  interview  protocols  and  topic  guides 
as  well  as  to  elicit  specific  information.  These 
personal  interviews  were  conducted  using  two 
topic  guides  (Appendix  I)-  One  topic  guide  was 
designed  for  interviewing  an  industry  group  or 
association;  the  other  was  targeted  for  an 
advertising  agency  representing  the  group  or 
association.  In  addition,  the  topic  guides  were 
intended  to  elicit  both  direct  answers  and 
secondary  information.  The  secondary  informa¬ 
tion  was  used  to  corroborate  the  direct  answers. 


For  instance,  in  the  case  of  Communications 
Workers  of  America  (CWA),  to  check  for 
consistency,  actual  media  decisions  were 
compared  to  its  stated  criteria. 

Telephone  interviews  were  then  conducted 
with  representatives  of  the  remaining  15  organi¬ 
zations.  Overall,  the  level  of  cooperation  of  all 
respondents  interviewed  was  excellent 

CHARACTERIZING  COLLABORATIVE 
ADVERTISERS 

It  became  apparent  early  in  the  research  effort 
that  simple  characteristics  like  a  particular  industry 
structure  and  an  industry-wide  interest  in  market 
expansion  or  cost  reduction  are  insufficient  to 
distinguish  collaborative  from  competitive  adver¬ 
tisers. 

Commodity,  non-brand  industries  with  many 
small  members  do  not  adequately  define  collabo¬ 
rative  advertisers.  -  For  example,  small  pork 
fanners  collectively  advertise  while  highly 
differentiated,  huge  mutual  fund  organizations 
also  collaborate  in  market  expansion  advertising. 
Conversely,  Florida's  fruit  and  vegetable  growers 
formed  a  cooperative,  but  they  do  not  collaborate 
on  advertising  because  an  adequate  representation 
of  the  interests  of  27  commodities  as  a  group  or  in 
part  is  difficult  Clearly,  a  particular  market 
structure  fails  to  isolate  industry  from  firm- 
specific  advertisers. 

Similarly,  a  common  interest  in  market 
expansion  advertising  does  not  necessarily 
sufficiently  unify  an  industry  to  act  collectively. 
Such  advertising  is  meant  to  expand  the  entire 


market  and/or  to  reduce  the  cost  of  doing 
business.  Yet  some  industries  collaborate  in 
advertising  while  others  do  not  In  addition, 
some  industry  members  use  the  market  expan¬ 
sionary  goals  of  industry  advertising  without 
collaborating  with  other  industry  members.  For 
example,  AT&T  ("Reach  out  and  touch  some¬ 
one")  and  Campbell's  Soup  ("Soup  is  Good 
Food")  each  conduct  market-expansion  adver¬ 
tising  campaigns  without  collaboration  with  other 
potential  competitors  in  their  industry.  Not  only 
must  other  conditions  exist  to  allow  the  collabo¬ 
ration  to  occur,  but  also  industry  members  must 
have  a  mutual  interest  in  the  same  type  of  market 
expansion.  As  noted  in  the  literature  review, 
expansion  campaigns  can: 

•  Generally  educate  the  consumer  as  to  the 
benefits  of  a  product 

•  Combat  an  enemy  that  distributes  mis¬ 
information  about  the  product 

•  Expand  the  product's  selling  season 

•  Suggest  new  uses  for  the  product 

•  Build  an  overall  image/emotion  for  the 
product 

Members  of  an  industry  must  share  an  interest  in 
a  particular  type  of  market  expansion  for 
cooperative  advertising. 

In  general,  industry  members  will  support 
collaborative  advertising  in  situations  where  they 
cannot  or  will  not  advertise  individually.  As 
noted  above,  these  situations  will  not  be  defined 
by  a  particular  market  structure  or  a  general 
industry-wide  interest  in  market  expansion. 
Instead,  a  specific  mutual  interest  must  be 
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established  among  industry  members  or  must 
already  exist  as  a  result  of  cooperation  in  other 
industry  functions  (trade  shows,  standard  setting, 
government  lobbying,  new  product  research, 
etc.)*  Then,  industry  members  can  weigh  the 
benefits  and  costs  of  industry-wide  cooperation  in 
advertising. 

FIVE  MOTIVATIONS  FOR  INDUSTRY 
ADVERTISING 

Given  a  common  interest  in  a  particular  type 
of  market  expansion  advertising  and  in  lowering 
the  costs  of  doing  business,  an  industry  may  have 
a  unique  set  of  reasons  for  deciding  to  advertise 
collectively.  In  fact,  the  industry  groups  WARC 
interviewed  did  articulate  various  subsets  of 
reasons,  but,  taken  together,  they  described  the 
following  five  major  motivations  for  collaboration 
in  advertising: 

•  To  take  advantage  of  economies  of  scale 
in  advertising 

•  To  address  secondary  target  groups  that 
are  different  from  those  they  could  reach 
individually 

•  To  address  the  primary  target  group  from 
a  different  perspective  than  could  be 
achieved  individually 

•  To  eliminate  '’free-riders"  (industry  mem¬ 
bers  that  benefit  without  paying)  resulting 
from  using  generic  message  content 

•  To  stimulate  goodwill  and  mutual  support 
within  die  entire  industry 


Economies  Of  Scale 

The  most  common  reason  given  for  an 
industry  uniting  to  advertise  collectively  is 
economies  of  scale.  As  long  as  the  industry 
members  share  a  common  message,  an  associa¬ 
tion  can  purchase  a  larger,  more  professional  ad 
campaign  than  its  members  can  individually.  For 
example,  the  American  Florists  Marketing 
Council  was  established  by  small,  independent 
florists  throughout  the  United  States.  As  a  group, 
these  florists  receive  the  benefits  of  quality 
posters  and  countercards,  and  a  national  adver¬ 
tising  campaign  for  Grandparents  Day.  Dearly, 
none  of  these  florists  could  have  individually 
afforded  these  advertisements.  The  benefit  of 
reduced  advertising  production  cost  unified  the 
florist  industry. 

In  addition,  collective  action  enables  member 
firms  to  reduce  media  costs  in  advertising. 
Frequently,  members  do  not  possess  the 
infrastructure,  skill  or  time  to  handle  advertising 
on  their  own.  In  such  cases  they  pool  their  media 
expertise  (e.g..  Linen  Promotion  Council, 
Savings  and  Loan  Foundation)  so  as  to  make 
more  effective  use  of  their  advertising  dollars. 

Another  economies  of  scale  distinction 
between  collaborative  advertising  and  firm-  or 
brand- specific  advertising  is  campaign  scope. 
Advertising  on  a  joint  basis  may  take  place  on  a 
national  scale  whereas  individual  firms 
concentrate  on  the  geographical  areas  of  their 
operations.  This  is  typically  the  case  with 
agencies  such  as  the  American  Gas  Association 
and  the  Investment  Company  Institute.  In  various 
cases,  reprints  of  national  advertisements  have 
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been  effectively  used  by  member  companies  to 
promote  their  products  in  their  own  markets. 

Finally,  collaborative  advertising  also  creates 
a  valuable  option  for  the  industry  members  based 
on  economies  of  scale.  When  an  industry  is 
performing  poorly,  the  industry  members  have 
the  option  to  increase  advertising  expenditures. 
In  contrast,  individual  advertisers  often  reduce 
advertising  expenses  in  a  failing  industry. 
Because  each  member  has  a  relatively  small 
vested  interest  in  the  campaign  and  the  potential 
benefits  of  improving  industry  sales  exist,  mem¬ 
bers  see  an  advantage  to  increasing  collaborative 
advertising.  For  example.  Investment  Company 
Institute,  an  assocation  of  mutual  funds,  launched 
a  $5  million  dollar  campaign  after  mutual  funds 
lost  $60  million  to  money  market  competition. 
The  mutual  funds  opted  to  contribute  incremental 
dollars  to  advertising  their  common  message 
against  a  common  threat 

Economies  of  scale  effects  were  most  fre¬ 
quently  cited  by  fragmented  industries  as  a  reason 
for  industry  collaboration.  These  results  may  be 
interpreted  two  ways.  Industries  with  many 
participants  selling  a  commodity  or  non-branded 
product  may  collaborate  because  product  differen¬ 
tiation  among  industries'  members  is  not 
possible.  More  concentrated  industries  (oligopo¬ 
lies  and  monopolies)  may  avoid  collaboration 
because  differentiation  is  paramount.  On  the 
other  hand,  the  U.S.  regulatory  environment  may 
preclude  collaboration  in  oligopolies,  and  as  a 
result,  only  fragmented  industries  collectively 
may  advertise  to  gain  economies  of  scale. 


Since  the  Department  of  Defense  is  not  subject 
to  anti-trust  regulation,  economies  of  scale  may 
support  its  use  of  Joint  advertising.  To  the  extent 
that  commonality  exists  among  the  Services  as  to 
what  the  military  offers  a  potential  recruit.  Joint 
advertising  may  allow  production  cost  savings. 
Similarly,  coordinating  all  media  purchases 
through  one  channel  or  agency  may  generate 
media  cost  savings. 

However,  the  perceived  requirement  for 
Service  differentiation  may  preclude  effective 
Joint  advertising.  Generic  Joint  advertising  to 
reduce  media  and  promotion  costs  of  advertising 
may  not  satisfy  the  individual  Services' 
requirements  as  effectively  as  Service-specific 
advertising.  Therefore,  the  potential  recruits' 
differential  response  to  Joint  and  Service-specific 
advertising  must  be  measured  to  determine  if  the 
economies  of  scale  and  effectiveness  of  Joint 
advertising  outweigh  the  effectiveness  of  Service 
advertising.  Though  potential  economies  of  scale 
seem  to  be  a  clear  reason  for  fragmented 
industries  to  collectively  advertise,  it  may  not 
necessarily  be  a  sufficient  reason  for  the 
Department  of  Defense  to  advertise  jointly. 
Nevertheless,  the  achievement  of  scale  economies 
can  be  hypothesized  for  Joint  advertising. 

Reaching  Secondary  Target  Groups  With 
the  Same  Message 

In  certain  cases,  industry  members  advertise 
collectively  because  as  a  group  they  can  reach 
audiences  that  in  turn  will  influence  the  behavior 
of  the  primary  target  group.  In  other  words, 
collaborative  advertising  can  be  a  more  effective 
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means  of  indirect  advertising.  For  example,  the 
12  local  chapters  of  the  Communication  Workers 
of  America  cannot  effectively  make  their  indivi¬ 
dual  interests  known  to  key  politicians  in 
Washington.  By  deciding  to  act  together  on  key 
common  issues,  they  are  able  to  project  a  united 
front  and  to  better  reach  national  legislators. 
CWA  hopes  that  this  presentation  of  a  national 
platform  will  cause  legislators  to  influence  AT&T 
management  toward  accepting  CWA's  demands. 

Similarly,  Joint  advertising  may  be  hypothe¬ 
sized  to  be  more  effective  than  Service-specific 
advertising  in  reaching  audiences  that  will 
influence  young  Americans  to  enlist  A  Joint 
campaign  designed  to  enhance  positive  attitudes 
about  military  service  by  highlighting  the  common 
benefits  of  all  the  Services  may  more  readily 
appeal  to  the  general  population.  Parents,  friends 
and  guidance  counselors  may  then  be  motivated  to 
encourage  members  of  the  primary  target  group  to 
consider  contacting  a  military  recruiter.  In  addi¬ 
tion,  this  indirect  advertising  may  encourage 
direct  enlistment  by  non-target  group  individuals. 
Hence,  Joint  advertising  may  be  hypothesized  to 
encourage  more  enlistments  among  older  indivi¬ 
duals  or  among  others  not  in  the  primary  target 
market. 

Reaching  The  Primary  Target  Group 
From  A  New  Perspective 

A  number  of  industry  groups  perceived 
collaborative  advertising  to  be  more  credible  than 
individual  firm  advertising.  The  representatives 
of  the  financial  service  industries  in  particular  felt 
this  to  be  the  case.  For  instance,  Investment 
Company  Institute  cited  improved  credibility  as  a 


major  reason  for  collaborative  advertising.  In 
many  cases  the  goal  of  collaborative  advertising 
was  described  as  the  "creation  of  an  appropriate 
image."  Therefore,  advertising  collectively  en¬ 
abled  firms  to  take  a  different  posture  when 
advertising.  In  other  words,  attitudinal  changes 
were  perceived  to  be  the  foremost  aim  of  such 
advertising.  However,  members  would  generally 
agree  to  contribute  to  the  budget  for  such 
advertising  only  when  they  were  reasonably 
convinced  of  their  ability  to  build  upon  it  and  take 
advantage  of  it  Several  instances  were  reported 
where  specific  campaigns  were  undertaken  to 
address  a  specific  problem.  Collaborative  adver¬ 
tising  was  used  as  one  tool  among  several  others 
to  address  the  situation.  In  the  context  of  the 
Department  of  Defense,  one  can  hypothesize  that 
Joint  advertising  is  perceived  as  more  credible 
among  certain  segments  of  the  youth  population 
and  that  it  is  more  effective  in  enhancing  the 
image  of  the  military  and  improving  basic 
attitudes  toward  enlistment. 

Eliminating  Free  Riders 

Eliminating  a  free  rider  problem  can  also 
induce  industry  members  to  advertise  collectively. 
The  free  rider  problem  usually  arises  in  a  non- 
branded,  commodity  industry  where  product 
differentiation  among  industry  members  is 
difficult.  An  individual  firm  using  a  generic 
message  in  advertising  inadvertently  benefits  its 
competitors.  So,  one  firm  is  paying  for  others  to 
receive  advertising  benefits  and  non-advertisers 
are  receiving  benefits  without  paying.  This 
externality  is  called  the  free-rider  problem.  Some 
collaborative  mcustry  advertisers  reported  that 
collaborative  advertising  reduced  or  eliminated 


this  problem  because  it  provided  a  mechanism  by 
which  all  industry  members  contribute  to  the 
advertising  campaign.  The  Florida  orange 
growers,  for  example,  suffered  free-rider  and 
quality  control  problems.  Not  only  did  non¬ 
advertisers  obtain  the  benefits  of  generic  orange 
advertising,  but  growers  of  poor-quality  oranges 
also  received  the  benefits  of  the  induced  demand 
for  high  quality  fruit.  Because  Florida  oranges 
were  not  brand-names,  consumers  could  not 
distinguish  advertised  high-quality  oranges.  As  a 
result,  the  growers  asked  the  state  to  regulate 
quality  standards  and  tax  all  growers  (on  a  per 
crate  basis)  to  fund  generic  Florida  orange 
advertising.  Collaborative  advertising  substan¬ 
tially  mitigated  the  free-rider  problem. 

In  contrast  to  non-branded,  commodity  indus¬ 
tries,  the  free-rider  problem  may  not  discourage 
the  use  of  the  generic  messages  by  individual 
firms  in  monopolistic  or  single  dominant  player 
industries.  For  example,  Campbell's  Soup  adver¬ 
tises  the  generic  message  -  "Soup  is  Good 
Food."  Since  Campbell  dominates  the  soup 
industry,  it  ignores  the  spillover  benefits  of  its  ads 
for  other  soup  manufacturers.  Similarly,  AT&T 
advertises  to  "Reach  Out  and  Touch  Someone" 
despite  free-riders. 

Joint  advertising  may  obviate  a  free-rider 
situation  which  may  be  hypothesized  to  arise 
when  one  service  conducting  a  generic  advertising 
campaign  (e.g.,  "Get  Experience")  may  benefit 
other  Services.  Collaborative  advertising 
removes  the  need  to  reallocate  budgets  among  the 
Services  to  account  for  spillover  effects. 


However,  it  is  not  clear  whether  a  free-rider 
problem  can  arise  at  the  Department  of  Defense. 
The  Department  of  Defense  consists  of  four 
competitive,  "branded"  Services.  If  the  Services 
are  sufficiently  differentiated  from  one  another,  a 
free-rider  problem  will  not  occur  when  a  Service 
uses  a  generic  message:  all  the  benefits  of 
advertising  will  be  accrued  by  the  advertiser. 
Spillover  effects  may  be  insufficient  to  require 
Joint  advertising. 

Instead,  Joint  advertising  may  remove  a 
different  externality  than  advertising  spillover.  In 
fact,  it  may  control  the  extent  to  which  the 
Services  can  differentiate  themselves  from  one 
another.  If  the  Department  of  Defense  allowed 
only  Service-specific  advertising,  the  Services 
may  concentrate  their  advertising  on  gaining 
market  share  rather  than  on  expanding  the  market 
The  Services  may  differentiate  themselves  to  such 
an  extent  as  to  be  dysfunctional.  Hence,  one  can 
hypothesize  that  less  Service  differentiation 
occurs  with  more  Joint  advertising. 

Generating  Goodwill 

Collaborative  advertising  was  reported  as 
contributing  to  goodwill  within  an  industry. 
Seeing  advantageous  advertising  makes  industry 
members  "feel  good"  about  the  industry.  For 
example,  CWA  members,  reading  their 
bargaining  positions  in  local  newspapers,  felt 
more  confident  about  their  requests. 

Similarly,  Joint  advertising  may  have 
substantial  goodwill  effects  for  the  Department  of 
Defense.  It  may  be  hypothesized  to  differentially 


encourage  recruiters  (one  of  the  most  influential 
factors  in  recruiting)  to  improve  their  effort. 
Furthermore,  Joint  advertising  may  help  stimulate 
reenlistment,  build  reserves,  and  smooth  the 
budgetary  process. 


may  produce  these  goodwill  effects.  Joint 
advertising  may  achieve  them  more  effectively. 


The  effectiveness  of  Joint  advertising  versus 
Service-specific  advertising  will  depend  upon 
differential  response  of  military  personnel  and  the 
external  audience  to  collaborative  and  individual 
advertising.  While  Service-specific  advertising 


SUMMARY  AND  CONCLUSION 

Hypotheses  for  evaluating  Joint  versus 
Service-specific  advertising  can  be  developed 
based  on  the  five  motivations  for  collaborative 
advertising  identified  in  the  industry  survey. 
Table  3.2  presents  a  schematic  representation  of 


survey  responses. 


Table  3.2 

REPORTED  MOTIVATIONS  FOR  COLLABORATIVE  ADVERTISING 
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Of  the  19  organizations  interviewed,  14 
indicated  multiple  motives  for  collaborative  adver¬ 
tising. 

The  five  objectives  identified  have  been 
categorized  into  economic  factors  (efficiency 
related)  and  marketing  factors  (effectiveness 
related).  Table  3.3  presents  a  summary  of  this 
categorization. 

The  weightings  for  economic  (efficiency)  and 
marketing  (effectiveness)  objectives  implied  for 


Table  33 

CATEGORIES  OF  OBJECTIVES  FOR  COLLABORATIVE  ADVERTISING 
Reported  Motivation  Number  of  Respondents 


the  sample  of  respondents  were  equal.  In 
summary,  although  this  research  effort  is  very 
exploratory  in  nature  and  not  amenable  to 
rigorous  statistical  testing  or  inferences,  at  least 
the  two  dimensions  of  economic  efficiency  and 
marketing  effectiveness  are  indicated  as  potential 
criteria  by  which  to  evaluate  the  Joint  advertising 
components  of  the  Advertisng  Mix  Test  These 
dimensions  are  comparable  to  and  consistent  with 
objectives  for  brand-  or  firm-specific  advertising. 


Economic  Motives  Only  S 

Marketing  Motives  Only  3 

Both  Economic  and  Marketing  Motives  9 

Total  with  Some  Economic  Motives  14 

Total  with  Some  Marketing  Motives  14 

Total  Respondents  9 
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CHAPTER  4 

CONCEPTUAL  MODELS  OF  MILITARY  RECRUITING 
ADVERTISING  BUDGETS 


This  chapter  presents  models  and  observa¬ 
tions  reported  by  Ambar  G.  Rao  in  "A  Model  for 
Joint  and  Service  Advertising  Budgets"  (October 
1983)  and  by  Hau  L.  Lee  in  "A  Generalized 
Model  for  Joint  and  Individual  Service  Adver¬ 
tising  on  Enlistment  for  the  Armed  Forces" 
(November  1983). 


Conceptual  models  of  advertising  contribu¬ 
tions  to  the  enlistment  process  were  developed  to 
gain  insights  about  advertising's  role  in  that 
environment  as  contrasted  to  the  product  sales 
processes  treated  in  the  literature.  Although  these 
models  were  not  intended  for  estimation,  several 
important  concepts  derived  from  the  exercise  are 
reflected  in  the  evaluation  criterion  and  candidate 
evaluation  measures: 


Repeat  purchases,  an  important  aspect  of 
product  sales,  is  a  negligible  feature  of  the 
enlistment  process. 


Service  quotas  and  quality  standards 
promote  "brand  switching"  among  the 
Services. 


•  The  primary  target  group  is  narrow  and 
highly  transitory. 


The  importance  of  the  enlistment  decision 
can  encourage  discussions  with  influ¬ 
ences  who  may  therefore  constitute  an 
important  advertising  target. 


Military  enlistment  has  a  strictly  sequential 
buying  cycle. 


An  intermediate  purchase  intention  can  be 
hypothesized  to  precede  actual  enlistment 
and  may  be  useful  for  predicting  subse¬ 
quent  behavior. 


Generally  speaking,  the  models  hypothesize 
two  kinds  of  contributions  from  military 
recruiting  advertising:  a  direct  contribution  to 
closing  enlistment  contracts  and  indirect 
contributions  in  improved  attitudes  toward 
military  service  among  youths  of  prime  enlistable 
age  (18-20  years),  youths  approaching  enlistable 
age  (15-17  years)  and  influencers  of  these 
populations  (parents,  guidance  counselors,  etc.). 
The  models  minimize  the  sum  of  Service-specific 
and  Joint  advertising  expenditures  while  assuring 
that  each  Service  meets  its  enlistment  contract 
requirements  across  time. 
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RAO’S  ONE  PERIOD  MODEL 

Rao  advances  a  simple  one  period  advertising 
model  under  the  following  hypotheses. 

HI  -  Advertising  Makes  Both  Short-Term 
and  Long-Term  Contributions 

Advertising  improves  the  attitude  toward 
military  service  of  18-20-year-old  youths. 
Improved  attitudes  among  this  group 
increases  the  size  of  the  pool  of  enlistable 
youths  who  are  favorably  disposed  to 
military  service.  Advertising  improves  the 
attitude  toward  military  service  of  IS- 17- 
year-old  youths  and  of  their  influencers. 
Improved  attitudes  among  these  groups 
increases  the  size  of  future  pools  of  high- 
quality,  18-20-year-old  youths  who  are 
favorably  disposed  to  military  service. 

H2  -  Enlistment  Contracts  to  a  Given 
Service  Arise  from  Three  Sources 

Enlistment  contracts  may  come  from  the 
pool  of  favorably  disposed  youths.  These 
contracts  are  denoted  "pool  contracts." 

Enlistment  contracts  may  arise  from  "targets 
of  opportunity"  outside  the  pool  of  favor¬ 
ably  disposed  youths  because  of  circum¬ 
stances  such  as  unemployment  These  con¬ 
tracts  are  denoted  "opportunity  contracts." 


H3  -  Enlistment  Requirements  for  All 
Services  Are  to  be  Met  at  the 
Smallest  Overall  Advertising  Cost 

Since,  in  his  one-period  model,  Rao  does 
not  distinguish  between  pool  contracts  and 
opportunity  contracts,  he  represents  the 
enlistment  demand  constraints  as 

r,  (A*)  +  g(AJ)K,(A*)  +  IE,  h,. ,  2>  R, 

i'm 

if  the  Service  obtains  less  than  its  require¬ 
ments  from  its  own  applicants. 

r,(A»)  +  g(Ai)Kf(A*)-Et  =  Rs 

if  a  Service  enjoys  a  surplus  of  applicants. 

where  s  e  {A.N.F.M}  denotes  Service  (Army, 
Navy,  USAF,  USMC,  respectively). 

R,  is  the  enlistment  requirement  of 

Service  s. 

A*  is  the  advertising  budget  of  Service  s. 

AJ  is  the  budget  for  Joint  advertising. 

r,(A*)  is  the  enlistment  contribution  of 
Service-specific  advertising.  Adver¬ 
tising  contributions  to  closing  con¬ 
tracts  and  to  improving  attitudes  are 
aggregated  in  this  factor.  rs(0)  >  0; 
i.e.,  even  in  the  absence  of  any 
Service-specific  advertising,  some 
contracts  would  be  realized. 


Enlistment  contracts  may  arise  from  surplus 
applications  to  others  Services.  These 
overflow  contracts  may  be  either  self- 
directed  or  guided  by  the  system. 


g(AJ)  is  the  aggregate  enlistment  contribu¬ 
tion  of  Joint  adveitising. 


hFtA  is  large)  or  for  heavy  Army-specific 
adveitising  if  not 


KS(A*)  is  the  share  of  g(AJ)  which  accrues  to 
Service  s.  K  is  represented  as  a 
function  of  the  advertising  budget  for 
Service  s.  K,(0)  >  0;  i.e.,  even  in  the 
absence  of  any  Service-specific  adver¬ 
tising,  some  contracts  would  be 
realized  as  a  result  of  Joint  adver¬ 
tising. 


is  the  surplus  of  applicants  to  Service 
s'. 


is  the  proportion  of  E,<  that  switches 
to  Service  s.  h  is  also  taken  to  be  a 
function  of  the  advertising  budget  for 
Service  s. 


Considering  only  two  Services,  the  Army  and 
the  Air  Force,  and  an  environment  where  the  Air 
Force  is  enjoying  a  surplus  of  applicants  as 
opposed  to  an  inadequate  supply  for  the  Army, 
Rao's  simple  one  period  model  becomes: 


minimize  Aa  +  Af  +  Aj 


subject  to: 


rA(AA)  +  g(AJ)KA(AA)  +  Ef  hFA  2  RA 


rF(AP)  +  g(A')KF(AF)  -  Ef  =  Rf 


From  this  structure,  Rao  sees  a  justification 
for  heavy  Air  Force  advertising  if  applicants  can 
be  efficiently  converted  to  Army  contracts  (i.e.,  if 


RAO'S  MULTI-PERIOD  MODEL 

After  observing  that  an  operational  model 
should  consider  4-6  periods  and  all  four  Services, 
Rao  advances  an  illustrative  model  for  two 
Services  and  two  time  periods. 


minimize  AAt  +  AFt  +  AJt  +  AAt+1  + 
AFt+i  +  AJt+i 


subject  to: 

PttA(AtA)  +  Ptg(AJt)KA(AAt)  + 

EF.t  •  hFAt  ^  Ryu 


+  Pt  gCA^KpCAF,)  -  Ef  =  RFft 


Pt+l  -  btm(AJ,AA,AF)  *  0 


where  s  e{A,N,F,M}  denotes  Service  (Army, 
Navy,  USAF,  USMC,  respectively). 


R*4  is  the  enlistment  requirement  of 
Service  s  in  period  L 


A*, 


is  the  advertising  budget  of  Service  s 
in  period  t. 


A', 


is  the  budget  for  joint-Services 
advertising  in  period  t 


r,(A*t)  is  the  enlistment  contribution  of 
Service-specific  advertising  in  period 
t.  Except  for  the  time  dimension,  r  is 
interpreted  as  in  the  single  period 
model. 
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g(AJt)  is  the  aggregate  enlistment  contribu¬ 
tion  of  Joint  advertising  in  period  t 

K,(A»,)  is  the  share  of  g(A;J  which  accrues 
to  Service  s  in  period  t  K  is  inter¬ 
preted  as  in  the  single  period  model. 

E,  tt  is  the  surplus  of  applicants  to  Service 

s'  in  period  t 

hi  A,  is  the  proportion  of  Ef  t  that  switches 
to  Service  s  in  period  t  Except  for 
the  time  dimension,  h  is  interpreted  as 
in  the  single  period  model. 

Pt  is  the  pool  of  18-20-year-olds  who 
are  favorably  disposed  to  military  ser¬ 
vice  in  the  year  t  (indexed  to  a  base 
year). 

b,  is  the  pool  of  15- 17-year-olds  who 
are  favorably  disposed  to  military 
service  in  the  year  t  (indexed  to  the 
same  base  year  as  Pt). 

Pt+l  -  blm(AJ,AA,AF):  i.e.,  to  pool  of 
favorably  disposed  18-20-year-olds 
depends  upon  advertising  previously 
absorbed  by  15-17-year-olds. 

Pt  scales  the  advertising  response  function.  If 
the  pool  of  favorably  disposed  18-20-year-olds 
were  to  remain  constant,  the  scaling  factor  would 
be  unity.  As  the  pool  increases  or  decreases,  the 
scaling  factor  varies  accordingly. 


From  this  formulation,  Rao  concludes  that  if 
Service-specific  and  Joint  advertising  expendi¬ 
tures  increase  Pt,  then  they  must  have  positive 
values.  In  particular,  if  m(AJ,AA,AF)  is  approxi¬ 
mated  by  m(AJ),  then  there  is  a  rationale  for  Joint 
advertising  regardless  of  its  direct  impact  on 
contracts.  Support  for  Service-specific  adver¬ 
tising  also  exists  when  it  increases  the  fraction  of 
the  target  population  including  a  Service  having 
unmet  enlistment  requirements  in  their  set  of  job 
alternatives. 

LEE'S  EXTENSIONS  OF  THE  RAO 
MODELS 

Lee  reviewed  and  developed  extensions  to  the 
Rao  models  in  "A  Generalized  Model  for  Joint 
and  Individual  Service  Advertising  on  Enlistment 
Contracts  for  the  Armed  Forces"  (November 
1983).  Extensions  advanced  by  Lee  include: 

•  A  full  specification  of  the  models  for  all 
Services. 

•  Relaxation  of  Rao's  assumption  that  Air 
Force  applicants  are  in  surplus  and  can  be 
accommodated  by  unmet  Army  require¬ 
ments. 

•  Differentiation  between  "pool"  and  "oppor¬ 
tunity"  contracts. 

•  Refinement  of  the  aging  progression. 

•  Differentiation  between  the  advertising 
responses  of  populations  which  are  favor¬ 
ably  disposed  and  those  which  arc  not. 
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Besides  these  refinements,  Lee's  treatment  of 
the  youth  population  differs  slightly  from  that  of 
the  Rao  models.  Lee  considers  17-21-year-olds 
to  be  the  prime  enlistable  population  and  14- 16- 
year-olds  to  be  the  population  approaching  enlist¬ 
ment  age. 

LEE'S  ONE  PERIOD  MODEL 

Since  Lee  distinguishes  between  "poor  and 
"opportunity"  contracts  ami  assumes  no  direction 
for  the  flow  of  surplus  applicants,  his  demand 
constraints  take  the  following  form: 

pV.CA*)  +  gf,(AJ)]  +  ptr\(A«)  +  g\(AJ)]  + 
X  h,  .(A)E*'  -  E* «  R»,  for  aU  s 

i'm 

where  p*  is  the  population  of  17-21-year-olds 
who  are  favorably  disposed  toward  military 
service. 

pa  is  the  population  of  17-21-year-olds 
who  are  not  favorably  disposed 
toward  military  service. 

s  €  {A,N,F,M}  denotes  Service  (Army, 
Navy,  US  AF,  USMC,  respectively). 

A*  is  the  advertising  expenditure  for 
Services. 

AJ  is  the  advertising  expenditure  for  joint- 


r*f(A»)  is  the  advertising  response  rate  of  pf 
for  Service  s:  i.e.,  r*g(A»)  is  the 
proportion  of  the  favorably  disposed 
population  who  join  as  a  result  of  A*. 

r*t(A*)  is  the  advertising  reponse  rate  of  p“ 
for  Service  s. 

gf,(AJ)  is  the  advertising  response  rate  of  pf 
for  Joint  advertising. 

gSCA*)  is  the  advertising  response  rate  of  p* 
for  Joint  Services. 

hj^A)  is  the  overflow  rate  as  a  proportion  of 
would-be  enlistees  to  Service  s  who 
will  enlist  in  Service  s'  if  A  are  the 
Service-specific  advertising  expendi¬ 
tures. 

E*’  is  the  overflow  from  Service  s’. 

R»  is  the  enlistment  requirement  for 
Service  s. 

The  following  constraints  are  required  to  pre¬ 
vent  the  model  genmating  more  enlistments  than 
persons  in  the  enlistable  population: 

gfg(AJ>  +  X  ^(A*)  $  1,  and 
s 


Services  advertising. 


gVAJ)  +  XtS(A»)£l. 


A  **  (AA,AN,AF,AM) 
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Also,  to  prevent  distributing  more  overflow  enlist¬ 
ments  than  persons  in  the  overflow  populations, 
the  following  constraint  is  imposed  for  every 
Service  s: 


^h,^  <,  1. 


Lee's  one  period  model  is  simply, 

minimize  AJ  +  AA  +  AN  +  AF  +  AM 

subject  to  the  foregoing  constraints. 

LEE'S  MULTI-PERIOD  MODEL 

A  major  consideration  in  multi-period  models 
is  the  transition  between  age  categories.  In  an 
annual  model,  not  all  the  14-16-year-olds  will 
transition  to  the  17-21-year-old  category.  Lee 
recognizes  this  difficulty  and  after  observing  that 
census  projections  might  resolve  the  matter  more 
exactly,  elects  to  assume  a  uniform  yearly  age 
distribution  within  the  age  categories.  Conse¬ 
quently,  one-fifth  of  the  17-21-year-old  pool  ages 
out  of  consideration  each  (annual)  period.  These 
individuals  are  replaced  by  one-third  of  the 
individuals  in  the  14-  16-year-old  pool.  The  14- 
16-year-old  pool  is  replenished  by  an  influx  of 
individuals  just  aging  into  consideration. 

Lee's  view  of  the  transitions  of  14- 16-year- 
olds  is  depicted  in  Figure  4.1.  In  his  multi-period 
model,  an  individual  may  transition  to  being 
favorably  disposed  to  military  service,  but  not  the 
reverse:  i.e.,  once  favorably  disposed,  always 


favorably  disposed.  For  time  periods  t  and  t+1. 
Figure  4.1  can  be  expressed  algebraically  as 
follows: 

Bf,+i  -  Nft+1  +  (2/3)  [Bft  +  g(At,AJt,B*J  and 

BVi  -  N*t+1  +  (2/3)  [B\  -  g(A„AJt)] 

where  advertising  expenditures  are  now  sub¬ 
scripted  for  time, 

Bft  and  Bat  are,  respectively,  the  pools  of  14- 
16-year-olds  who  are  favorably 
disposed  to  military  service  and 
those  who  are  not 

Nft  and  N*t  are  new  additions  to  those  pools 
of  14- 16-year-olds. 


g(A,^t) 


is  the  conversion  rate  to  the  pool 
of  individuals  who  are  favorably 
disposed  toward  military  service 
from  the  pool  of  14- 16-year-old 
individuals  who  are  not. 


Transitions  for  17-21-year-olds  are  shown  in 
Figure  4.2.  The  transition  equations  implied  by 
that  figure  are  as  follows: 

Pf,+i=  (1/3)  [Bft  +  g(At,AJt)B*J  + 

(4/5)Pf,  +  m(At,AJl)P*l]  and 

P*,+l  =  (l/3)B*t  [1  -  g(A„A,l)]  + 

(4/5)P*,  [1  -  m(A„AJt)] 


1 

1 
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Figure  4.1 

Transitions  of  Age  Group  (14-16) 
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where  Pft  and  P»t  are,  respectively,  the  pools  of 
17-21-year-olds  who  are  favorably  disposed  to 
military  service  and  those  who  are  not. 

m(At^J,)  is  the  conversion  rate  for  17-21- 
year-olds  analagous  to 
for  14-16-year-olds. 

All  that  remains  to  complete  the  constraint  set 
for  the  multi-period  model  is  the  enlistment 
demand  constraints.  These  constraints  are  similar 
to  those  for  the  single  period  model  and  for  each 
Service  s  and  time  period  t  can  be  expressed  as 
follows: 

Pft[rf,(A*t)  +  gfs(AJt)]  + 

P*t[r*,(A*t)  +  g*s(AJt)]  + 

ZhSff(At)E*t-E*t«R*t 


where  response  rates  r  and  g  are  as  defined  for  the 
one  period  model. 

Lee's  multi-period  model  can  be  expressed  in 
simplified  form  as 

minimize  X  (AJt  +  AA,  +  ANt  +  AFt  +  AMt) 

subject  to:  the  enlistment  demand  constraint 
in  every  period,  14- 16-year-old 
population  transition  constraints, 
17-21-year-old  population  transi¬ 
tion  contraints,  non-negativity  of 
advertising  expenditures. 


as 
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CHAPTER  5 
CONCLUSIONS 


The  research  reported  here  has  focused  on 
developing  a  criterion  by  which  one  advertising 
treatment  of  the  FY  1984  Advertising  Mix  Test 
can  be  compared  with  another.  This  criterion  pro¬ 
vides  the  basis  to  compare  level  of  advertising 
expenditures  and  the  mix  of  the  expenditures 
between  collaborative  (Joint)  advertising  and 
individual  Service  advertising. 

A  review  of  the  relevant  literature,  interviews 
with  commercial  collaborative  advertisers,  and  the 
process  of  developing  conceptual  models  of 
military  enlistment  advertising  effects  have 
strongly  supported  use  of  economic  efficiency 
and  program  effectiveness  criteria. 

These  criteria  can  be  applied  both  to  firm-  or 
brand-level  advertising  and  to  collaborative 
advertising.  Accordingly,  evaluation  of  the  FY 
1984  Advertising  Mix  Test  will  use  a  compound 
criterion:  The  advertising  policy  of  choice 
will  be  one  providing  the  necessary  short¬ 
term  and  long-term  contributions  to  the 
attainment  of  the  Services'  enlistment 
requirements  and  having  the  lowest  total 
advertising  budget. 

Although  a  treatment  of  measures  of 
effectiveness  and  analytical  methods  will  be 
provided  in  a  subsequent  report,  some  prelimi¬ 
nary  measures  of  effectiveness  and  methodologies 
are  previewed  in  the  following  sections. 


A  WORD  ABOUT  MEASURES 

A  number  of  measures  for  evaluating  the 
experimental  results  will  be  derived  from  data 
collected  as  part  of  the  experiment  Measures 
under  consideration  are  either  aggregate  sales 
response  measures  or  aggregated  intermediate 
measures  with  demonstrated  reliability  and 
predictive  validity.  The  measures  are  based  on 
observed  behavior,  on  reported  behavior,  and  on 
attitudes.  Measures  of  observed  behavior  include 
observation  of  applicants  and  contracts  by  Service 
and  by  educational/test  attainment  Measures  of 
reported  behavior  include  reported  recruiter 
contacts  and  reported  conversations  regarding 
enlistment  Attitudinal  measures  include 
statements  of  intention  to  join  the  military  or  a 
specific  Service.  These  measures  focus  on 
different  phases  of  the  strictly  sequential 
enlistment  cycle.  This  large  variety  of  measures 
is  appropriate  because  each  individual  measure 
has  potential  limitations  which  may  be 
substantially  alleviated  if  the  measures  are  used 
together  in  a  consensus  form  of  policy  evaluation. 

A  WORD  ABOUT  METHODOLOGY 

The  observed  enlistment  behavior  measures  of 
contracts  and  applicants  will  be  analyzed  using 
actual  monthly  observations  of  advertising  and  the 
observed  behavior  of  contracts  and  applicants  for 
each  market.  Regression-based  analysis  tech¬ 
niques  are  appropriate  for  these  measures  and  will 
allow  the  investigation  of  lags  in  advertising 
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response  and  of  effects  of  recruiting  phenomena 
such  as  the  number  of  recruiters  in  a  given 
market,  the  size  of  the  Delayed  Entry  Program, 
and  recruiting  missions. 

The  reported  behaviors  and  attitudinal 
measures  are  derived  from  questions  in  the  Youth 
Attitude  Tracking  Study.  Since  this  study  is 
administered  in  the  fall  of  each  year,  only  two 
observations  will  be  available  for  each  market  or 
treatment  cell.  For  these  measures,  cross¬ 


sectional  analysis  of  variance  comparisons  of  the 
measures  and  their  rates  of  change  across 
treatment  conditions  are  appropriate. 

Each  advertising  strategy  (treatmentcondition) 
will  be  evaluated  using  the  same  measures.  These 
evalutions  will  be  compared  for  consistency  and 
every  effort  will  be  made  to  explain  and  reconcile 
any  conflicting  indications.  Recommendations 
regarding  the  level  and  mix  of  DoD  advertising 
will  be  based  on  a  consensus  across  evaluations. 


8 


V  / 


9 WJliJI  WTJ  VVV  WWVV.V.tot.  v;»-j  rvYxrwi  «r.  uni 


RPBWWWWWJWW1.MJW 


■WTF^TKJr^  IT*  »  *  "  *  -J* 


El 


t 


f* 


APPENDIX  1 


INDUSTRY  ADVERTISING  TOPIC  5. 
GUIDE 

We  are  undertaking  a  research  project 
concerned  with  the  impact  of  industry  advertising. 

As  part  of  the  project,  we  are  examining  the 
practices  used  in  developing,  executing  and 
evaluating  industry-oriented  advertising  as  differ¬ 
entiated  from  brand  or  company-oriented  adver¬ 
tising  campaigns.  Your  association  has  been 
selected  for  participation  in  the  study  because  of 
its  industry  promotion  strategy.  Accordingly,  we 
would  like  to  ask  you  a  few  questions  about  this 
program  and  how  it  was  developed  and  evaluated. 

1.  When  was  the  campaign  first  initiated? 

What  prompted  the  development  of  an 
industry  campaign?  What  were  its  initial 
objectives?  What  is  the  target  population 
(i.e.,  customers,  workers  in  the  industry, 
regulators)? 

2.  How  has  the  campaign  developed  over 
time?  How  has  it  grown?  Why? 


What  are  the  current  elements  of  the 
campaign  strategy  and  how  large/important 
is  each  element? 

•  public  relations  activity  (e.g.,  news 
articles,  news) 

•  speeches 

•  media  and  advertising  (break  out  budget 
by  proportion) 

—  newspaper 

—  magazines 

—  TV 

—  radio 

—  direct  mail 

—  telemarketing 

—  other 

6.  How  is  media  mix  determined? 

7.  How  many  people  are  involved  in  planning 
and  executing  the  strategy?  Who  are  they? 
Which  member  companies  are  involved? 
What  are  their  roles  and  what  are  they  trying 
to  get  out  of  it? 


3.  What  are  the  current  goals  and  objectives  of 
the  campaign?  How  is  this  strategy 
different  from  that  used  by  companies 
within  the  industry?  What  are  the  tradeoffs 
and  conflicts  between  industry  campaigns 
and  company  specific  campaigns  (e.g., 
coordination,  conflict  over  strategies,  etc.)? 


8.  Who  decides  on  creative  strategy? 

9.  Are  there  any  constraints  placed  on  the 
campaign  ads  by  the  member  firms?  Are 
there  any  constraints  placed  on  member 
company  ads  by  the  industry? 


4.  Who  determines  the  budget?  How? 
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10.  Is  there  an  advertising  agency  involved? 
Who  handles  agency  relations?  Would  you 
mind  if  we  contacted  the  agency?  Could 
you  provide  us  with  a  point  of  contact? 


11.  How  is  advertising  success/effectiveness 
measured?  Who  determines  whether  the 
campaign  is  successful  or  needs  revision 
(e.g.,  the  association,  member  companies)? 
Is  there  regular  feedback  on  the  campaign 
from  member  companies? 


12.  Is  there  a  program  of  market  research 
carried  out  in  conjunction  with  the  program? 
How  is  the  research  used?  What  questions 
does  it  answer?  What  criteria  are  used  to 
assess  the  impact  of  the  campaign?  How 
large  is  the  budget? 


13.  What  have  been  the  major  benefits  and 
achievements  of  the  strategy?  What  have 
been  the  major  problems? 


14.  Has  the  campaign  been  particularly  useful  or 
particularly  detrimental  to  one  set  of  member 
companies  or  customers?  Why? 


15.  For  the  future- will  the  strategy  be 
enhanced,  kept  the  same,  decreased?  Why? 


16.  If  you  were  to  set  the  advertising  budget, 
would  you  increase  it,  decrease  it,  or  keep  it 
at  the  same  level?  Why? 


17.  How  would  you  define  industry  adver¬ 


tising" 


ADVERTISING 

GUIDE 


AGENCY 


TOPIC 


We  aie  undertaking  a  research  project  con¬ 
cerned  with  the  impact  of  industry  advertising. 
As  part  of  the  project,  we  are  examining  the  prac¬ 
tices  used  in  developing,  executing  and  evaluating 
industry-oriented  advertising  as  differentiated 
from  brand  oi  company-oriented  advertising 
campaigns.  Your  agency  has  been  selected  for 
participation  in  the  study  because  of  its  industry 
promotion  strategy.  We  have  been  referred  to 

you  by _ .  We 

would  Like  to  ask  you  a  few  questions  about 
industry  advertising  programs  conducted  by  the 
agency  and  how  they  have  been  developed  and 
evaluated. 


How  many  industry-oriented  campaigns  has 
the  agency  undertaken  in  the  last  three  to 
five  years?  For  what  kinds  of  organi¬ 
zations?  Have  you  handled  campaigns  for 
member  companies  as  well  as  for  the 
industry  in  question? 


What  have  been  the  goals  and  objectives  of 
these  campaigns  (develop  examples)?  What 
is  the  audience-customers,  regulators,  the 
industry  itself?  What  prompted  these 
campaigns  to  be  undertaken? 


2a.  Are  there  tradeoffs  and  conflicts  between 
industry  campaigns  and  company  specific 
campaigns?  What  are  the  key  areas  of 
tradeoff  and  conflict? 
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3.  Who  is  involved  in  decision-making 
regarding  the  campaigns?  In  associations 
and  industry  trade  groups?  In  government 
agencies? 

4.  How  are  budgets  set  for  the  campaigns  by 
the  various  clients?  How  large  are  the  bud¬ 
gets  relative  to  product-specific  or  company- 
specific  campaigns? 

5.  What  media  mix  is  used  (give  rough 
proportional  breakouts,  if  posable)  for  3/4 
major  campaigns? 

•  TV 

•  radio 

•  newspaper 

•  magazines 

•  direct  mail 

•  telemarketing 

•  other 

6.  How  (i.e.,  on  what  basis)  is  the  media  mix 
determined? 

7.  Are  the  media  schedules  used  differently 
from  those  used  for  product-  specific 
advertising? 

8.  How  is  creative  strategy  developed  and 
implemented?  Is  it  different  from  product  or 
company-specific  advertising?  How? 

8a.  Do  member  companies  get  involved  in  any 
of  the  above? 


9.  How  are  the  accounts  managed?  How  is  it 
different  from  the  management  of  product- 
specific  accounts? 

9a.  Are  there  any  restrictions  placed  on 
campaign  ads  by  member  firms  or  on 
member  company  ads  by  the  industry? 

10.  How  do  the  goals  of  the  industry  campaign 
overlap  or  diverge  from  the  goals  of 
company-specific  campaigns? 

1 1.  How  is  advertising  effectiveness/campaign 
success  measured?  How  is  this  different 
from  that  used  for  product-specific 
campaigns? 

11a.  Do  you  perceive  any  industry  campaign  to 
have  been  particularly  useful  or  particularly 
detrimental  to  any  subset  of  member 
companies  or  subset  of  customers? 

12.  Is  marketing  research  carried  out  in 
conjunction  with  these  campaigns?  What 
questions  does  the  research  address?  What 
criteria  are  used  to  assess  impact? 

13.  If  you  were  to  set  the  optimal  advertising 
level  for  the  industry  campaigns  you  have 
named,  would  you  increase/decrease  or 
keep  it  the  same?  Why?  What  would  you 
do  with  additional  funds? 


APPENDIX  F 


TESTING  FOR  SIGNIFICANCE: 

A  "DIFFERENCE  OF  DIFFERENCES"  ESTIMATOR 


Within  each  test  cell  (Yellow,  Blue,  Green,  and  Red),  we  observe  responses  to  the  YATS  question¬ 
naire  at  two  points  in  time,  1983  and  1984.  The  proportion  of  respondents  in  cell  i  answering  "yes"  to  a 
particular  survey  question  in  year  t  is  defined  as  pu. 

Since  pi  84  and  pi83  are  the  result  of  responses  to  the  same  survey  question  measured  at  two  points  in 

time,  the  measure  dt  =  p^M  -  p,  is  interpreted  as  an  estimator  of  the  change  in  the  (true)  population 
proportion  answering  "yes"  across  time. 


Since  it  represents  an  estimator  of  the  difference  between  two  population  proportions,  dt  is  distributed 

as  Student's  t.  Since  the  individual  pi4  are  binomially  distributed,  the  variance  of  their  difference  is 
given  by 


Var(di)  =  Var(p.84-piM) 


(Pi^d-Pup) 


N, 


i.83 


(Pi.84)  *  Pi, 84^ 


N. 


i.84 


where  is  the  size  of  the  sample  in  cell  i  in  year  t  Typically,  the  above  expression  would  include  a 
term  involving  the  covariance  between  p^M  and  p^83,  but  since  the  pi  t  are  the  results  of  independent 

YATS  samples  taken  in  successive  years,  they  are  statistically  independent  and  a  covariance  term  does 
not  appear  in  the  above  expression. 


We  now  proceed  to  compute  whether  the  difference  between  any  pair  of  differences  d4  and  dj  is  itself 
significantly  different  from  zero.  For  example,  the  difference  in  the  1983  to  1984  time-difference 
between  cells  i  and  j  is  denoted  by  and  is  given  by 

D.  =  d  -d 
u  »  j 

In  our  analysis,  we  are  particularly  interested  in  this  measure  where  i  is  one  of  the  treatment  cells  Blue, 
Green,  or  Red,  and  j  represents  the  Yellow,  or  control,  cell.  The  variance  of  Dt.  is,  as  above,  the  sum  of 
the  variances  of  d,  and  d-: 


Since  the  ceils  represent  independent,  nonoverlapping  populations,  the  covariance  between  d;  and  d.  is 
zero,  and  the  above  expression  contains  no  covariance  term. 

Since  the  dL  are  themselves  distributed  as  Student's  t,  their  difference  D-  is  also  t-distributed  and  a  test 
of  the  null  hypothesis  that  the  true  difference  *  0  is  performed  by  evaluating  the  test  statistic: 

t  =  Dy  lyfv ar(Dy) 

If  the  absolute  value  of  t  exceeds  the  two-tailed  critical  value  (based  on  the  significance  level  of  the  test 
being  performed),  we  conclude  that  is  significantly  different  from  zero.  Where  this  is  so,  the  change 
across  time  is  significantly  greater  in  one  cell  than  another,  a  phenomenon  that  is  attributed  to  the 
effects  of  advertising  in  the  particular  cells. 


APPENDIX  G 


STATISTICAL  MODELS  USED  TO  TEST 
THE  RELATIONSHIP  BETWEEN  ENLISTMENT 
BEHAVIOR  AND  ADVERTISING 


APPENDIX  G 


Statistical  Models  Used  to  Test  the  Relationship 
between  Enlistment  Behavior  and  Advertising 


The  discussion  in  Chapter  7  noted  that  the 
analyses  undertaken  found  no  statistically 
significant  relationship  between  enlistment 
behavior  and  either  the  level  of  total  adver¬ 
tising  or  the  proportion  allocated  to  Joint 
advertising. 

Table  1  summarizes  the  findings  of  ten  cross- 
sectional  regression  models  developed  to 
explore  this  relationship.  In  general,  the 
analyses  revealed  no  consistent  pattern  in  the 
relationship  between  advertising  and  various 
categories  of  applicants.  The  regression 
analyses  did,  however,  uncover  two 
numerically  small  but  still  statistically 
significant  relationships  between  high  school 
graduate  and  senior  applicant  rates  in  general 
and  in  the  upper  three  test  categories 
specifically,  and  the  proportion  of  total 
advertising  allocated  to  the  Joint  program. 

Nevertheless,  it  appears  that  the  most  plausi¬ 
ble  explanation  for  advertising's  effect  on 
enlistment  behavior  (i.e.,  applicant  rates)  is  to 
assume  an  interaction  effect  exists  between  the 
level  of  total  advertising  and  the  proportion 
allocated  to  the  Joint  program.  That  is,  for 
any  given  level  of  total  spending,  there  is  a 
proportion  allocated  to  Joint  representing  the 
optimum  combination  that  yields  the  maxi¬ 
mum  enlistment  response.  (Refer  to  Chapter 
7  fora  further  discussion  of  this.) 


Design  of  the  Models 

As  mentioned  earlier,  the  models  used  to  test 
these  relationships  were  formulated  using 
cross-sectional  regression  methods.  More 
specifically  the  generalized  model  can  be 
described  as: 

Log  (ApplicantSjj/Populationi) 

=  b„  Log  (PJOINTi) 

+  b2iLog(TOTADVi) 

+  b*  Log  (UNEMPj) 

+  b*  Log  (URBAN;) 

+  bSi  Log  (RACEj) 

+  bfi  Log  (RECRj) 

Since  the  log  form  of  the  models  has  been 
used,  the  coefficients  for  the  respective  pre¬ 
dictor  variables  can  be  interpreted  directly  as 
elasticities.  The  full  models  employed  all  72 
ADI  markets,  while  the  reduced  models 
excluded  those  13  ADI  markets  that 
composed  cell  Blue.  This  was  done  in  order 
to  determine  whether  cell  Blue  introduced  any 
anomalous  effects  in  the  model.  The  results  in 
Table  1  showed  that  cell  Blue's  behavior  was 
similar  to  the  pattern  observed  for  all  cells  in 
the  full  model. 
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REGRESSION  ANALYSIS 
GRAPHS  OF  DUMMY  VARIABLES 
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This  appendix  provides  details  of  the  regression  analysis  supporting  the  conclusion  that  an 
advertising  mix  with  a  higher  proportion  of  joint  advertising  and  a  relatively  lower  absolute  total 
dollar  expenditure  per  capita  on  Joint  and  Service-specific  advertising  is  the  most  optimal. 

A  two  dimensional  chart  of  total  advertising  per  capita  and  percentage  joint  advertising  for  the  72 
markets  in  the  advertising  experiment  is  constructed.  An  example  of  such  a  chart  is  shown 
overleaf.  The  chart  is  then  subdivided  into  cells  defined  according  to  the  levels  of  total  per  capita 
advertising  and  percentage  joint  advertising.  This  chart  is  divided  into  nine  cells  and  six  cells  as 
shown. 

To  demonstrate  that  the  conclusions  are  not  sensitive  to  the  demarcation  of  cells,  two  nine  cell  sub¬ 
divisions  and  two  six  cell  sub-divisions  are  constructed  as  shown.  To  each  cell  a  corresponding 
dummy  variable  is  assigned  such  that  it  takes  a  value  equal  to  one  if  a  market  belongs  to  the  cell 
and  zero  otherwise.  There  are  eight  dummies  for  the  nine  cell  sub-division  and  five  dummies  for 
the  six  cell  sub-division  since  the  effect  of  one  cell  is  included  in  the  regression  constant  term. 

The  regression  analysis  was  conducted  with  applicants  and  contracts  (by  quality  level)  as  the 
dependent  variable  and  the  dummies  in  addition  to  the  standard  covariates  recruiters  per  capita, 
unemployment  and  racial  composition.  It  is  observed  that  the  dummy  variables  corresponding  to 
cells  with  low  total  advertising  and  higher  percentage  joint  advertising  are  significant.  In  some 
cases  the  dummy  variables  corresponding  to  cells  with  high  total  advertising  and  high  percentage 
joint  advertising-  are  significantly  negative,  implying  that  such  an  advertising  mix  might  be 
counterproductive. 
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Dummy  Variable  Definitions: 

T  =  Total  Expenditure;  J  =  %  Joint  Expenditure 
All  Dummy  Variables  D2  to  D9  are  set  to  zero  initially.  Each  dummy  is  set  to  1  when: 


D2»  1 
D3  =  1 
D4=  1 
D5»  1 
D6  =  1 
D7  =  1 
D8  ■  1 
D9  *  1 


(T  below  $5.5)  and  (J  between  20%  and  60%) 

(T  below  $5.5)  and  (J  between  60%  and  100%) 

(T  between  $5.5  and  $7.0)  and  (J  below  20%) 

(T  between  $5.5  and  $7.0)  and  (J  between  20%  and  60%) 
(T  between  $5.5  and  $7.0)  and  (J  above  60%) 

(T  above  $7.0  and  (J  below  20%) 

(T  above  $7.0)  and  (J  between  20%  and  60%) 

(T  above  $7.0)  and  (J  above  60%) 
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Dummy  Variable  Definitions: 

T  =  Total  Expenditure;  J  «  %  Joint  Expenditure 

All  Dummy  Variables  D2  to  D9  are  set  to  zero  initially.  Each  dummy  is  set  to  1  when: 

D2=l  (T  below  $5.0)  and  (J  between  15%  and  40%) 

D3  «  1  (T  below  $5.0)  and  (J  between  40%  and  100%) 

D4  *  1  (T  between  $5.0  and  $7.0)  and  (J  below  15%) 

D5  =  1  (T  between  $5.0  and  $7.0)  and  (J  between  15%  and  40%) 

D6  =  1  (T  between  $5.0  and  $7.0)  and  (J  above  40%) 

D7  »  1  (T  above  $6.5)  and  (J  below  1 5%) 

D8  =  1  (T  above  $6.5)  and  (J  between  15%  and  40%) 

D9  *  1  (T  above  $6.5)  and  (J  above  40%) 
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TOTAL  $  ADJPOP.  vs.  %  JOINT  AD. 
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Dummy  Variable  Definitions: 

T  »  Total  Expenditure;  J  =  %  Joint  Expenditure 

All  Dummy  Variables  D2  to  D6  are  set  to  zero  initially.  Each  dummy  is  set  to  1  when: 

D2  -  1  (T  below  $6.0)  and  (J  below  30%) 

D3  »  1  (T  below  $6.0)  and  (J  between  30%  and  60%) 

D4  »  1  (T  at  or  above  $6.0)  and  (J  below  30%) 

D5  *  1  (T  at  or  above  $6.0)  and  (J  between  30%  and  60%) 

D6  ■  1  (T  at  or  above  $6.0)  and  (J  above  60%) 
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TOTAL  $  ADJPOP.  vs.  %  JOINT  AD. 
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Dummy  Variable  Definitions: 


T  =  Total  Expenditure;  J  *  %  Joint  Expenditure 
All  Dummy  Variables  D2  to  D6  are  set  to  zero  initially.  Each  dummy  is  set  to  1  when: 


D2-1 
D3-1 
D4-1 
D5-1 
D6«  1 


(T  below  $6.5)  and  (J  below  20%) 

(T  below  $6.5)  and  (J  between  20%  and  40%) 

<T  at  or  above  $6.5)  and  (J  below  20%) 

(T  at  or  above  $6.5)  and  (J  between  20%  and  40%) 
(T  at  or  above  $6.5)  and  (J  above  40%) 
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adi  liii-im 

TELEVISION  MAMET  OODB/NAMB  LIST 
(nuaerical) 


ADI 

CODE 

ADI  NAME 

ADI 

CODE 

3 

BOSTON 

135 

9 

NEW  YORK 

139 

11 

PHILADELPHIA 

140 

13 

LOS  ANGELES 

141 

15 

SAN  DIEGO 

143 

17 

SANTA  BARBARA- SANTA  MARIA-SA 

145 

19 

WASHINGTON,  DC 

147 

21 

BALTIMORE 

149 

23 

SALISBURY 

151 

25 

HARTFORD-NEW  HAVEN 

153 

29 

PITTSBURGH 

155 

31 

YOUNGSTOWN 

157 

33 

JOHNSTOWN-ALTOONA 

159 

35 

CLEVELAND 

161 

39 

EL  CENTRO-YUMA 

165 

43 

HARRISBURG- YORK-LANCASTER-LE 

173 

45 

SPRINGFIELD,  MA 

175 

47 

PROVIDENCE-NEW  BEDFORD 

177 

51 

CHICAGO 

179 

53 

SOUTH  BEND- ELKHART 

181 

55 

TOLEDO 

183 

57 

DETROIT 

185 

59 

GRAND  RAP IDS- KALAMAZOO- B ATT L 

187 

61 

LANSING 

195 

63 

FLINT-SAGINAW-BAY  CITY 

197 

65 

SAN  FRANCISCO 

199 

67 

SACRAMENTO-STOCKTON 

201 

69 

SALINAS-MONTEREY 

203 

71 

FRESNO— VISALIA 

205 

73 

BAKERSFIELD 

207 

75 

ST.  LOUIS 

209 

77 

SPRINGFIELD-DECATUR-CHAMPAIG 

211 

83 

INDIANAPOLIS 

213 

85 

LAFAYETTE,  IN 

215 

87 

TERRE  HAUTE 

217’ 

89 

CHICO-REDDING 

219' 

91 

FT.  WAYNE 

221 

93 

CINCINNATI 

225 

95 

DAYTON 

227 

101 

LIMA 

229 

103 

WHEELING-STEUBENVI LLE 

231 

105 

SEATTLE-TACOHA 

233 

107 

MINNEAPOLIS-ST.  PAUL 

235 

109 

DALLAS- FT.  WORTH 

237 

111 

MILWAUKEE 

241 

113 

MADISON 

243 

115 

WAUSAU-RHINELANDER 

245 

117 

LA  CROSSE-EAU  CLAIRE 

247 

119 

ROCKFORD 

249 

121 

COLUMBUS,  OH 

251 

123 

PORTLAND- POLAND  SPRING 

253 

125 

ZANESVILLE 

255 

127 

MIAMI 

257 

129 

WEST  PALM  BEACH- FT.  PIERCE-V 

259 

131 

TAMPA-ST.  PETERSBURG 

261 

133 

FT.  MYERS-NAPLES 

263 

ADI  NAME 


BUFFALO 
ROCHESTER,  NY 
ELMIRA 
SYRACUSE 

WILKES  BARRE-SCRANTON 

BINGHAMTON 

ERIE 

ALBANY-SCHENECTADY-TROY 
BURL I NGTON- PLATTSBURGH 
WATERTOWN-CARTHAGE 
UTICA 

KANSAS  CITY 
ST.  JOSEPH 
PRESQUE  ISLE 

ROCHESTER-MASON  CITY-AUSTIN 
CEDAR  RAPIDS-WATERLOO— DUBUQU 
PEORIA 

DAVENPORT-ROCK  ISLAND-MOLINE 

MEMPHIS 

NASHVILLE 

JACKSON,  TN 

HUNTSVI LLE-DECATUR-FLORENCE 

PADUCAH-CAPE  GIRARDEAU-HARRI 

BOWLING  GREEN 

ATLANTA 

CHATTANOOGA 

HOUSTON 

AUSTIN,  TX 

WACO-TEMPLE 

EVANSVILLE 

LOUISVILLE 

LEXINGTON 

GREENVILLE-SPARTANBURG-ASHEV 

KNOXVILLE 

BRISTOL-KINGSPORT-JOHNSON  Cl 
MACON 

BIRMINGHAM 

SELMA 

QUINCY- HANNIBAL 

COLUMBIA- JEFFERSON  CITY 

TUSCALOOSA 

PORTLAND,  OR 

EUGENE 

MEDFORD 

DENVER 

COLORADO  SPRINGS-PUEBLO 
NEW  ORLEANS 
BEAUMONT- PORT  ARTHUR 
BATON  ROUGE 
LAKE  CHARLES 
LAFAYETTE,  LA 
ALEXANDRIA,  LA 
CHARLESTON- HUNTINGTON 
PARKERSBURG 
CLAR  KSBURG-WESTOM 
OKLAHOMA  CITK 
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TELEVISION  MARKET  OODB/NAMB  LIST 
(numerical) 


AOI  ADI  NAME 

CODE 

265  ARDMORE-ADA 

269  TULSA 

271  SAN  ANTONIO 

273  LAREDO 

275  PHOENIX 

277  TUCSON 

279  CHARLOTTE 

281  GREENSBORO-WINSTON  SALEM-HIG 

283  NORFOLK— PORTSMOUTH-NEWPORT  N 

285  RICHMOND 

287  HARRISONBURG 

291  SALT  LAKE  CITY 

293  TWIN  FALLS 

295  IDAHO  FALLS-POCATELLO 

297  HELENA 

299  GREAT  FALLS 

301  OMAHA 

303  DES  MOINES 

305  OTTUMWA- K I RKSVILLE 

307  WICHITA-HUTCHINSON 

313  TOPEKA 

31 5  GREEN  BAY 

317  MARQUETTE 

319  LITTLE  ROCK 

321  SHREVEPORT-TEXARKANA 

323  TYLER 

325  FT.  SMITH 

327  MONROE-EL  DORADO 

329  ORLANDO-DAYTONA  BEACH-MELBOU 

331  LINCOLN-HASTINGS-KEARNEY 

335  JACKSONVILLE 

337  SPOKANE 

339  YAKIMA 

342  MISSOULA 

345  ROANOKE- LYNCHBURG 

3*»7  BLUEF I  ELD- BECK  LEY-OAK  HILL 

351  RALEIGH-DURHAN 

353  GREENVILLE-NEW  BERN-WASH I NGT 

355  WILMINGTON 

357  BANGOR 

359  FLORENCE,  SC 

361  COLUMBIA,  SC 

363  BILOXI -GULFPORT- PASCAGOULA 

367  ALBUQUERQUE 

369  ROSWELL 

371  EL  PASO 

373  JACKSON,  MS 

375  GREENWOOD-GREENVILLE 

377  MERIDIAN 

379  LAUREL-HATTI ESBURG 

381  DULUTH-SUPERIOR 

383  MOB1LE-PENSACOLA 

385  NORTH  PLATTE 

389  SIOUX  FALLS-HITCHELL 

391  SIOUX  CITY 

393  FARGO 


ADI  ADI  NAME 

CODE 

395  ALEXANDRIA,  MN 

403  AMARILLO 

405  WICHITA  FALLS-LAWTON 

409  COLUMBUS,  GA 

411  MONTGOMERY 

413  TALLAHASSEE-THOMASVILLE 

415  DOTHAN 

417  PANAMA  CITY 

419  ALBANY,  GA 

421  AUGUSTA 

423  CHARLESTON,  SC 

425  SAVANNAH 

427  SPRINGFIELD,  MO 

429  JOPLIN-PITTSBURG 

431  JONESBORO 

433  CORPUS  CHRISTI 

435  MCALLEN-BROWNSVILLE:  LRGV 

437  LUBBOCK 

439  ODESSA-MIDLAND 

441  ABILENE-SWEETWATER 

443  SAN  ANGELO 

445  BOISE 

448  COLUMBUS-TUPELC 

449  MANKATO 

451  TRAVERSE  CITY-CADILLAC 

455  LAS  VEGAS 

457  BILLINGS-HARDIN 

459  RENO 

460  GLENDIVE 

462  MI NOT- BISMARCK -DICKINSON 

465  CHEYENNE 

467  EUREKA 

469  RAPID  CITY 

471  CASPER-RIVERTON 

473  GRAND  JUNCTION-DURANGO 

5L3  VICTORIA 

577  PALM  SPRINGS 

591  BEND 

601  HAGERSTOWN 

603  ANNISTON 

613  BUTTE 

621  GAINESVILLE 

625  FLAGSTAFF 

627  ALPENA 

645  SARASOTA 

651  CHARLOTTESVILLE 


1-6 


TOrvurwarTtan^ji'n*  n»  tjfiui  r%m ; 


APPENDIX  J 

FY83  AND  FY84  CELL  MEANS 
ORIGINAL  AND  MODIFIED 
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APPENDIX  J 


FY83  MEANS 

(ORIGINAL*  CELL  MAPPING) 


PBTJ. 

YELLOW 

rP-fi- 

BLUE 

PCT.T- 

GREEN 

CELL 

RED 

TOTAL 

EXOGENOUS 

EMPLOYMENT 

0.1045 

0.0879 

0.0995 

0.1100 

0.1022 

RECRUITERS 

0.0146 

0.0145 

0.0152 

0.0150 

0.0147 

APPLICANTS 

TOTAL 

0.0S82 

0.0577 

0.0571 

0.0615 

0.0588 

HSG 

0.0495 

0.0487 

0.0481 

0.0515 

0.0497 

NONHSG 

0.0087 

0.0089 

0.0090 

0.0100 

0.0091 

i-mA 

0.0282 

0.0279 

0.0270 

0.0282 

0.0280 

mB-v 

0.0204 

0.0198 

0.0201 

0.0224 

0.0207 

CONTRACTS 

TOTAL 

0.0311 

0.0321 

0.0313 

0.0341 

0.0320 

NONHSG 

0.0025 

0.0030 

0.0029 

0.0029 

0.0028 

I-mA 

0.0189 

0.0193 

0.0188 

0.0202 

0.0193 

IHB-V  . 

0.0096 

0.0098 

0.0096 

0.0109 

0.0100 

ADVERTISING 

JOINT 

1.3661 

1.3117 

1.5271 

1.3726 

1.3795 

SERVICE-SPECIFIC 

3.7577 

3.8120 

4.2209 

4.3013 

3.9685 

*  Original  Dumpings  include  Hanrisburg-York,  Pennsylvania  in  Cell  Blue,  Grand  Junction, 
Colorado  inCell  Red 
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FY83  MEANS 

(MODIFIED*  CELL  MAPPING) 


MEANS 


CRT  J. 
YELLOW 

CELL 

BLUE 

CELL 

GREEN 

CELL 

RED 

TOTAL 

EXOGENOUS 

UNEMPLOYMENT 

0.1043 

0.0882 

0.0987 

0.1108 

0.1022 

RECRUITERS  MONTHS 

PER  17-21  YEAR  OLD  MALE 

0.0146 

•  0.0143 

0.0151 

0.0148 

0.0147 

APPLICANTS 

PER  17-21  YEAR  OLD  MALE 
TOTAL 

HSG  &  SENIORS 

NONHSG 

HSG  &  SENIORS,  I-HIA 

HSG  &  SENIORS,  DIB- V 

0.0587 

0.0501 

0.0086 

0.0286 

0.0205 

0.0579 

0.0487 

0.0092 

0.0277 

0.0200 

0.0555 

0.0468 

0.0088 

0.0265 

0.0194 

0.0597 

0.0502 

0.0095 

0.0272 

0.0222 

0.0588 

0.0497 

0.0091 

0.0280 

0.0207 

CONTRACTS 

PER  17-21  YEAR  OLD  MALE 
TOTAL 

HSG 

NONHSG 

HSG  &  SENIORS,  I-DIA 

HSG  &  SENIORS,  mB-V 

0.0316 

0.0291 

0.0025 

0.0193 

0.0098 

0.0318 

0.0287 

0.0030 

0.0190 

0.0097 

0.0302 

0.0273 

0.0029 

0.0181 

0.0091 

0.0331 

0.0304 

0.0027 

0.019S 

0.0109 

0.0320 

0.0293 

0.0028 

0.0193 

0.0100 

ADVERTISING 

JOINT  ($  PER  17-21  MALE) 
SERVICE-SPECIFIC 
($  PER  17-21  MALE) 

1.3823 

3.8073 

1.2983 

3.7281 

1.4793 

4.1476 

1.3390 

3.8809 

1.3795 

3.9685 

*  Modified  Cell  Mappings  include  Harrisburg- York,  Pennsylvania  in  Cell  Yellow  and  exclude 
Grand  Junction,  Colorado.  These  modifications  are  made  based  on 


advertising  during  the  test  year. 


actual  deliveries  of 
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P.E.P.  SYSTEMS,  INC. 

DOCUMENTATION  OF  MEDIA  ALLOCATION  METHODOLOGY 
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Jun*  16,  1986 


Prof.  Vincent  Carroll 
Wharton  Con tor  for  Applied  Research 
3508  Market  Stroot  Suite  100 
Philadelphia,  Pennsylvania  .19104 


Dear  Vinnie, 

1  am  forwarding  to  you  documentation  that  describes 
the  allocation  methodology  used  by  P.E.P.  Systems  in  the 
processing  of  data  for  the  Joint  Advertising  Mix  Test. 
This  pertains  to  all  National  Advertising  as  submitted 
under  Item  4  of  the  1984  Data  Call. 

If  there  are  any  questions  concerning  the  attached, 
please  call.  Otherwise,  have  a  great  summerl 


Sincerely, 


L . 


Phoebe  Wiener 


» 
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P.E.P  Systems  Inc.,  1270  Broadway,  New  York,  N  Y.  10001  •  ( 212)564-0333 


GEOGRAPHIC  UNIT  OF  ALLOCATION 


Geographic  market  areas  used  for  National  distribu¬ 
tion  are  the  Arbitron  defined  markets  known  as  the  Areas 
of  Dominant  Influence  or  A.D. I.  A.D.l.s  cover  the  total 
continental  United  States  contiguously  and  therefore  in¬ 
corporate  dll  3072  counties.  The  1962  configuration  is 
used  as  the  base  year  for  all  distribution. 


ALLOCATION  PROCESS 


By  month,  program,  service/ Joint . 

1. Network  TV  (including  Syndicated  programming!: 

Line-ups  for  each  show  -  data  by  station  elim¬ 
inated  for  black-out  adjustments.  Station 
share  (1)  to  allocate  show  dollars  and  impres¬ 
sions  to  each  station  for  station  allocation 
into  A.D.I.,  (account  for  spill.! 


2. Spot  TV  : 

Station  dollars  and  impressions  for  allocation 
into  A.D. 1 . 


3. Cable  TV  : 

Dollars  and  impressions  allocated  based  on  Cable 
System  Households/A. D. I . 
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PEP.  Systems  Inc.,  1270Broadway,  New  York,  N  Y.  10001  •(212)564-0333 


4.  Net  work.  Radio  (Line,  Syndicated,  Rep.  I : 

Line-ups  for  each  network  with  elimination  of 
black-out  stations.  Station  share  (II  to  al¬ 
locate  network  dollars  and  impressions  into 
A.D.l. 


S.Spot  Radio: 

Station  dollars  and  impressions  for  allocation 
into  A.D. 1 . 


6. Magazines  (National): 

Dollars  and  impressions  allocated  by  A.D.l. 
in  proportion  to  circulation#  distribution. 


7. Magazines  (Regional): 

Dollars  and  impressions  allocated  into  A.D.l. 
in  proportion  to  local  circulation#. 


8. Newspapers: 

Dollars  and  impressions  allocated  into  A.D.l. 
in  proportion  to  actual  circulation.# 


9. Outdoor: 

Dollars  and  impressions  allocated  directly  in¬ 
to  A.D.l.  of  appearance. 


10. Direct  Mail: 

Dollars  allocated  into  A.D.l.  in  proportion  to 
the  circulation  (impressions)  distributed  into 
A.D.l. 


#  Audit  Bureau  of  Circulation 
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ELECTRONIC 


MEDIA 


ALLOCATION 


DETAIL 


Network  Television  (including  Syndicated)  -  Dollars  and  Impressions 
.  (M  18-2*  Yrs I 

Show  line-ups  for  each  month. 


A  -  Households  per  station  within  show  line-up  calculated  so 

that  each  station  has  a  proportional  share  If )  of  the  total 
network  show  households  to  be  multiplied  by  show  cost/ 
impressions  per  month. 


B  -TV  station  dollars/impressions  IM  16-24  Yrs) 

are  then  distributed  by  A.D. 1.  using  Nielsen*  average 
viewing  hours  by  county  per  station. 


Cable  Television  -  Dollars  and  Impressions  (M  16-24  Yrs) 


Cable  System  line-ups  for  each  month. 


For  each  Cable  Station,  Household  subscribers  are  recorded 
by  ZIP  code  within  a  Cable  System.  ZIP  coded  Households 
are  recoded  into  A.D.l.  for  distribution  of  each  Cable 
System's  dollars/impressions. 


Spot  TV-  Dollars  and  Impressions  IM  18-24  Yrs) 


Dayparts  for  each  Station  per  month. 


T  V  station  do] 1 art/ impress ions  IM  18-24  Yrs  I 

are  distributed  by  A.D.l.  using  Nielsen*  average  viewing 

hours  by  county  per  station. 
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Network  Radio  -  Dollars  and  Impressions  (M  18-24  Yrs ) 


Network  line-ups  (one  or  more.)  for  each  month  by  Program. 


A  -  Males  18-24  Yrs  impressions  per  station  developed  for 

each  radio  network  line-up.  All  M  18-24  Yrs  impressions 
are  calculated  so  that  each  network  radio  station  has  a 
a  proportional  share  (1)  of  the  total  radio  network  im¬ 
pressions#  for  distribution  of  line-up  cost/total  impres¬ 
sions  for  each  month  for  station  dollars/impressions. 


B  -  Radio  Station  Dollars/Impressions  (M  18-24  Yrs) 

are  distributed  by  A.D.l.  using  Arbitron  listening  audience 
by  county  per  station. 


Spot  Radio  -Dollars  and  Impressions  (M  18-24  Yrs) 


Daypart s  for  each  station  per  month. 


A  -  Radio  Station  dollars/impressions  IM  18-24  Yrs I 
(See  Network  Radio  Part  B) 


«A.C.  Nielsen  Company  -  N  S  1  County/Coverage  Study 
Average  over  three  cycles  -  Nov.  1983,  Feb.  1984,  &  May  1984 


MEASUREMENT  &  RATING  SOURCES: 


Television 

Nielsen  Network  Programs  by  Designated  Market  Areas 
Metro  Areas  Totals  -  Station  Totals 

Nielsen  Report  on  Syndicated  Program  Audiences 

Nielsen  Market  Daypart  Summaries 

Nielsen  National  Television  Ratings 

Radio 

Arbitron  Radio  Audience  Estimate  by  Market 
Arbitron  Radio  County  Coverages 

Magazines 

MR1 -  Mediamark  Research  Inc.  Magazine  Audience 
Estimates 

Newspapers 

National  Simmons  Reader/Copy  (NAB I 
Outdoor 

National  Association  of  Outdoor  Advertisers 


